=== events in openshift-ingress (2026-10-30T22:16:51Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2 to ip-10-0-140-179.ec2.internal 37m Normal Scheduled pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh Successfully assigned openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 to ip-10-0-137-72.ec2.internal 50m Normal Scheduled pod/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r Successfully assigned openshift-ingress/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r to ip-10-0-142-3.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 to ip-10-0-142-3.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f to ip-10-0-137-72.ec2.internal 67m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 to ip-10-0-137-72.ec2.internal 64m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 to ip-10-0-142-3.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 to ip-10-0-140-179.ec2.internal 51m Normal Scheduled pod/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 Successfully assigned openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 to ip-10-0-142-3.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq to ip-10-0-140-179.ec2.internal 59m Normal Scheduled pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Successfully assigned openshift-ingress/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv to ip-10-0-140-179.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs to ip-10-0-140-179.ec2.internal 24m Normal Scheduled pod/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t Successfully assigned openshift-ingress/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t to ip-10-0-142-3.ec2.internal 67m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-cdm7z to ip-10-0-142-3.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq to ip-10-0-137-72.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z to ip-10-0-140-179.ec2.internal 29m Normal Scheduled pod/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 Successfully assigned openshift-ingress/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 to ip-10-0-142-3.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-h2mcv to ip-10-0-140-179.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-lkltg to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr to ip-10-0-140-179.ec2.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-bz96z to ip-10-0-137-72.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 to ip-10-0-142-3.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h to ip-10-0-140-179.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-mf2db Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-mf2db to ip-10-0-137-72.ec2.internal 64m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v to ip-10-0-140-179.ec2.internal 51m Normal Scheduled pod/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs Successfully assigned openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz Successfully assigned openshift-ingress/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz to ip-10-0-140-179.ec2.internal 48m Normal Scheduled pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Successfully assigned openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 to ip-10-0-142-3.ec2.internal 5m19s Normal Scheduled pod/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-5tf9b Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-5tf9b to ip-10-0-137-72.ec2.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 to ip-10-0-142-3.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-m9l2r to ip-10-0-140-179.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft to ip-10-0-142-3.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j to ip-10-0-137-72.ec2.internal 26m Normal Scheduled pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Successfully assigned openshift-ingress/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c to ip-10-0-142-3.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j to ip-10-0-137-72.ec2.internal 31m Normal Scheduled pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Successfully assigned openshift-ingress/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-ait-92d5d744-5c548bc679-rtwfr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-92d5d744-5c548bc679-rtwfr to ip-10-0-137-72.ec2.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-7kdrm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-7kdrm to ip-10-0-137-72.ec2.internal 67m Normal Scheduled pod/payload-processing-5f78687d5b-kjg7p Successfully assigned openshift-ingress/payload-processing-5f78687d5b-kjg7p to ip-10-0-140-179.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 to ip-10-0-137-72.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq to ip-10-0-137-72.ec2.internal 5m26s Normal Scheduled pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Successfully assigned openshift-ingress/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx to ip-10-0-140-179.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s to ip-10-0-140-179.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-be3ada-76ffbdb68-f8ntj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-be3ada-76ffbdb68-f8ntj to ip-10-0-142-3.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Successfully assigned openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr to ip-10-0-140-179.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-8964ac-976f957fd-vhvrt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-8964ac-976f957fd-vhvrt to ip-10-0-137-72.ec2.internal 65m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj to ip-10-0-140-179.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 to ip-10-0-140-179.ec2.internal 41m Normal Scheduled pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Successfully assigned openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb to ip-10-0-142-3.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm to ip-10-0-140-179.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69 to ip-10-0-140-179.ec2.internal 31m Normal Scheduled pod/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f to ip-10-0-140-179.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b to ip-10-0-140-179.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-b-f23eae-86bbf64bdb-cxtqq Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-f23eae-86bbf64bdb-cxtqq to ip-10-0-137-72.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk to ip-10-0-142-3.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-abf04b-c4db67488-6jzt4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-abf04b-c4db67488-6jzt4 to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh Successfully assigned openshift-ingress/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh to ip-10-0-140-179.ec2.internal 51m Normal Scheduled pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt Successfully assigned openshift-ingress/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt to ip-10-0-140-179.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql Successfully assigned openshift-ingress/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql to ip-10-0-137-72.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m to ip-10-0-140-179.ec2.internal 53m Normal Scheduled pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Successfully assigned openshift-ingress/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 to ip-10-0-140-179.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w to ip-10-0-137-72.ec2.internal 67m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-4nxfb to ip-10-0-140-179.ec2.internal 59m Normal Scheduled pod/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h Successfully assigned openshift-ingress/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h to ip-10-0-140-179.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl to ip-10-0-140-179.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq to ip-10-0-140-179.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4 to ip-10-0-140-179.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf Successfully assigned openshift-ingress/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf to ip-10-0-140-179.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls to ip-10-0-140-179.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2 to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 Successfully assigned openshift-ingress/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 to ip-10-0-142-3.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2tsvw to ip-10-0-142-3.ec2.internal 37m Normal Scheduled pod/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 Successfully assigned openshift-ingress/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 to ip-10-0-142-3.ec2.internal 64m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp to ip-10-0-142-3.ec2.internal 48m Normal Scheduled pod/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss Successfully assigned openshift-ingress/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss to ip-10-0-142-3.ec2.internal 40m Normal Scheduled pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Successfully assigned openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw to ip-10-0-140-179.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Successfully assigned openshift-ingress/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 to ip-10-0-142-3.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 to ip-10-0-140-179.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc to ip-10-0-140-179.ec2.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-ait-399049f3-77f856b49f-f24sl Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-399049f3-77f856b49f-f24sl to ip-10-0-137-72.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-2ff29 to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg to ip-10-0-142-3.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-api-a-0e7510-646866ffb7-t5vrz Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-0e7510-646866ffb7-t5vrz to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 Successfully assigned openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 to ip-10-0-140-179.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v to ip-10-0-140-179.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq to ip-10-0-137-72.ec2.internal 50m Normal Scheduled pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Successfully assigned openshift-ingress/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm to ip-10-0-140-179.ec2.internal 67m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx to ip-10-0-137-72.ec2.internal 50m Normal Scheduled pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Successfully assigned openshift-ingress/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx Successfully assigned openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx to ip-10-0-142-3.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv to ip-10-0-142-3.ec2.internal 71m Normal Scheduled pod/kube-auth-proxy-5649f7bc66-trj7m Successfully assigned openshift-ingress/kube-auth-proxy-5649f7bc66-trj7m to ip-10-0-137-72.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 to ip-10-0-137-72.ec2.internal 38m Normal Scheduled pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j Successfully assigned openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j to ip-10-0-142-3.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 Successfully assigned openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 to ip-10-0-137-72.ec2.internal 71m Normal Scheduled pod/kube-auth-proxy-5649f7bc66-klvr2 Successfully assigned openshift-ingress/kube-auth-proxy-5649f7bc66-klvr2 to ip-10-0-140-179.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-p2pvx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-p2pvx to ip-10-0-137-72.ec2.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nb8vx to ip-10-0-140-179.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-nkgpq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-nkgpq to ip-10-0-137-72.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p66fx to ip-10-0-140-179.ec2.internal 67m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz to ip-10-0-140-179.ec2.internal 8m20s Normal Scheduled pod/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 Successfully assigned openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 to ip-10-0-140-179.ec2.internal 12m Normal Scheduled pod/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 to ip-10-0-140-179.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 to ip-10-0-137-72.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw Successfully assigned openshift-ingress/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw to ip-10-0-137-72.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 to ip-10-0-142-3.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-p8tnk to ip-10-0-137-72.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f to ip-10-0-137-72.ec2.internal 53m Normal Scheduled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m Successfully assigned openshift-ingress/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m to ip-10-0-142-3.ec2.internal 56m Normal Scheduled pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Successfully assigned openshift-ingress/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv to ip-10-0-140-179.ec2.internal 50m Normal Scheduled pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Successfully assigned openshift-ingress/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d to ip-10-0-140-179.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 to ip-10-0-137-72.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-mt-c1da2183-8649468f78-r2ncj Successfully assigned openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-r2ncj to ip-10-0-137-72.ec2.internal 49m Normal Scheduled pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Successfully assigned openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl to ip-10-0-140-179.ec2.internal 58m Normal Scheduled pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Successfully assigned openshift-ingress/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq to ip-10-0-140-179.ec2.internal 33m Normal Scheduled pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Successfully assigned openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj to ip-10-0-142-3.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf to ip-10-0-142-3.ec2.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 to ip-10-0-140-179.ec2.internal 22m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-kzl4j Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-kzl4j to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt to ip-10-0-140-179.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg Successfully assigned openshift-ingress/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm to ip-10-0-142-3.ec2.internal 70m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-jggkz Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-jggkz to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-hd6sz Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hd6sz to ip-10-0-142-3.ec2.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-rh5rw to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-dtvjx to ip-10-0-137-72.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 to ip-10-0-137-72.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 Successfully assigned openshift-ingress/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 to ip-10-0-140-179.ec2.internal 53m Normal Scheduled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z Successfully assigned openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8s4mz to ip-10-0-142-3.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-7b98bb678c-tbwcl Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-tbwcl to ip-10-0-140-179.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Successfully assigned openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp to ip-10-0-137-72.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 to ip-10-0-140-179.ec2.internal 60m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr to ip-10-0-140-179.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg to ip-10-0-137-72.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc Successfully assigned openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc to ip-10-0-142-3.ec2.internal 52m Normal Scheduled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 Successfully assigned openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 to ip-10-0-140-179.ec2.internal 60m Normal Scheduled pod/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w Successfully assigned openshift-ingress/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w to ip-10-0-140-179.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q to ip-10-0-137-72.ec2.internal 63m Normal Scheduled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Successfully assigned openshift-ingress/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-bpjlj Successfully assigned openshift-ingress/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-bpjlj to ip-10-0-137-72.ec2.internal 72m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-47kpq Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-47kpq to ip-10-0-140-179.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-6sn9p Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-6sn9p to ip-10-0-137-72.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc to ip-10-0-140-179.ec2.internal 53m Normal Scheduled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 Successfully assigned openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 to ip-10-0-137-72.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-ll5l7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-ll5l7 to ip-10-0-137-72.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-jx66m Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-jx66m to ip-10-0-137-72.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r to ip-10-0-140-179.ec2.internal 72m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-wxstq to ip-10-0-142-3.ec2.internal 40m Normal Scheduled pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh Successfully assigned openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh to ip-10-0-140-179.ec2.internal 38m Normal Scheduled pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Successfully assigned openshift-ingress/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm to ip-10-0-142-3.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq to ip-10-0-140-179.ec2.internal 52m Normal Scheduled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 Successfully assigned openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 to ip-10-0-137-72.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk to ip-10-0-140-179.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn to ip-10-0-140-179.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn to ip-10-0-140-179.ec2.internal 29m Normal Scheduled pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Successfully assigned openshift-ingress/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt to ip-10-0-140-179.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm to ip-10-0-137-72.ec2.internal 5m19s Normal Scheduled pod/payload-processing-e2e-ait-6ad3224d-79cfdfb994-rnnj2 Successfully assigned openshift-ingress/payload-processing-e2e-ait-6ad3224d-79cfdfb994-rnnj2 to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/payload-processing-e2e-ait-92d5d744-779597976-7xjvh Successfully assigned openshift-ingress/payload-processing-e2e-ait-92d5d744-779597976-7xjvh to ip-10-0-140-179.ec2.internal 52m Normal Scheduled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk Successfully assigned openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk to ip-10-0-142-3.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 to ip-10-0-142-3.ec2.internal 21m Normal Scheduled pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Successfully assigned openshift-ingress/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj to ip-10-0-142-3.ec2.internal 64m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj to ip-10-0-137-72.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx Successfully assigned openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-api-a-59fb17-89bc69f67-56k2g Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-59fb17-89bc69f67-56k2g to ip-10-0-137-72.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-znkvj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-znkvj to ip-10-0-137-72.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb to ip-10-0-137-72.ec2.internal 53m Normal Scheduled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk Successfully assigned openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk to ip-10-0-140-179.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg to ip-10-0-137-72.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 to ip-10-0-137-72.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx to ip-10-0-140-179.ec2.internal 48m Normal Scheduled pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Successfully assigned openshift-ingress/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-h7dcj Successfully assigned openshift-ingress/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-h7dcj to ip-10-0-137-72.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs to ip-10-0-140-179.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk to ip-10-0-142-3.ec2.internal 54m Normal Scheduled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Successfully assigned openshift-ingress/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c to ip-10-0-140-179.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-api-a-0e7510-574b9c5d94-hvp2q Successfully assigned openshift-ingress/payload-processing-e2e-api-a-0e7510-574b9c5d94-hvp2q to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Successfully assigned openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 to ip-10-0-140-179.ec2.internal 38m Normal Scheduled pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 Successfully assigned openshift-ingress/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 to ip-10-0-142-3.ec2.internal 34m Normal Scheduled pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx Successfully assigned openshift-ingress/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx to ip-10-0-140-179.ec2.internal 64m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v to ip-10-0-140-179.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-wr2lz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-wr2lz to ip-10-0-142-3.ec2.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-57f78bcd55-xgns5 to ip-10-0-137-72.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq Successfully assigned openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq to ip-10-0-137-72.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Successfully assigned openshift-ingress/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp to ip-10-0-142-3.ec2.internal 57m Normal Scheduled pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn Successfully assigned openshift-ingress/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn to ip-10-0-140-179.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 to ip-10-0-137-72.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf Successfully assigned openshift-ingress/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf to ip-10-0-142-3.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h to ip-10-0-140-179.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-api-a-59fb17-649c79cc74-jrnbm Successfully assigned openshift-ingress/payload-processing-e2e-api-a-59fb17-649c79cc74-jrnbm to ip-10-0-137-72.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 Successfully assigned openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 to ip-10-0-137-72.ec2.internal 67m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz to ip-10-0-137-72.ec2.internal 61m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 to ip-10-0-142-3.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr to ip-10-0-137-72.ec2.internal 37m Normal Scheduled pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm Successfully assigned openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm to ip-10-0-142-3.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 to ip-10-0-137-72.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q to ip-10-0-137-72.ec2.internal 53m Normal Scheduled pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 Successfully assigned openshift-ingress/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 to ip-10-0-140-179.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn to ip-10-0-140-179.ec2.internal 61m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 to ip-10-0-140-179.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-f8137c-77958df654-xxgsq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f8137c-77958df654-xxgsq to ip-10-0-142-3.ec2.internal 54m Normal Scheduled pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq Successfully assigned openshift-ingress/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq to ip-10-0-140-179.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54 to ip-10-0-140-179.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg to ip-10-0-140-179.ec2.internal 24m Normal Scheduled pod/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 Successfully assigned openshift-ingress/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 to ip-10-0-142-3.ec2.internal 8m26s Normal Scheduled pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Successfully assigned openshift-ingress/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg to ip-10-0-140-179.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-api-a-ca18b3-668696fdfd-9w2s9 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-ca18b3-668696fdfd-9w2s9 to ip-10-0-137-72.ec2.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-derive-c906b806-9477cb87f-bgv72 Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-c906b806-9477cb87f-bgv72 to ip-10-0-137-72.ec2.internal 4m43s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj to ip-10-0-140-179.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-api-b-517366-86d679464c-rh6sb Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-517366-86d679464c-rh6sb to ip-10-0-137-72.ec2.internal 63m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm to ip-10-0-142-3.ec2.internal 12m Normal Scheduled pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Successfully assigned openshift-ingress/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf to ip-10-0-140-179.ec2.internal 52m Normal Scheduled pod/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv Successfully assigned openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv to ip-10-0-142-3.ec2.internal 23m Normal Scheduled pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Successfully assigned openshift-ingress/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl to ip-10-0-140-179.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf to ip-10-0-140-179.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Successfully assigned openshift-ingress/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 to ip-10-0-140-179.ec2.internal 56m Normal Scheduled pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Successfully assigned openshift-ingress/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl to ip-10-0-140-179.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq to ip-10-0-137-72.ec2.internal 63m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn to ip-10-0-137-72.ec2.internal 61m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 to ip-10-0-140-179.ec2.internal 61m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 to ip-10-0-142-3.ec2.internal 61m Normal Scheduled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Successfully assigned openshift-ingress/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-rflpm Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-rflpm to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-qdztb Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-qdztb to ip-10-0-137-72.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Successfully assigned openshift-ingress/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v to ip-10-0-142-3.ec2.internal 51m Normal Scheduled pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l Successfully assigned openshift-ingress/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l to ip-10-0-140-179.ec2.internal 71m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-669569fcd5lntp7 Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-669569fcd5lntp7 to ip-10-0-142-3.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 Successfully assigned openshift-ingress/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 to ip-10-0-140-179.ec2.internal 28m Normal Scheduled pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 Successfully assigned openshift-ingress/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 to ip-10-0-140-179.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db to ip-10-0-140-179.ec2.internal 12m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj to ip-10-0-137-72.ec2.internal 8m20s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh to ip-10-0-137-72.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw to ip-10-0-142-3.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp to ip-10-0-137-72.ec2.internal 52m Normal Scheduled pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Successfully assigned openshift-ingress/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s to ip-10-0-140-179.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-api-b-4266b7-6f95cbbb78-s4vfd Successfully assigned openshift-ingress/payload-processing-e2e-api-b-4266b7-6f95cbbb78-s4vfd to ip-10-0-137-72.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw to ip-10-0-142-3.ec2.internal 65m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 to ip-10-0-140-179.ec2.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc to ip-10-0-137-72.ec2.internal 27m Normal Scheduled pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Successfully assigned openshift-ingress/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh to ip-10-0-140-179.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-api-b-517366-7c5b497f8d-nrz2x Successfully assigned openshift-ingress/payload-processing-e2e-api-b-517366-7c5b497f8d-nrz2x to ip-10-0-137-72.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w to ip-10-0-137-72.ec2.internal 4m43s Normal Scheduled pod/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-skfxn Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-skfxn to ip-10-0-137-72.ec2.internal 67m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq to ip-10-0-140-179.ec2.internal 76m Normal Scheduled pod/router-default-6789dbc4b5-5gqgl Successfully assigned openshift-ingress/router-default-6789dbc4b5-5gqgl to ip-10-0-142-3.ec2.internal 77m Warning FailedScheduling pod/router-default-6789dbc4b5-5gqgl no nodes available to schedule pods 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq to ip-10-0-140-179.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 to ip-10-0-142-3.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk to ip-10-0-137-72.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-e136b8-c8c99b868-b7jb9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-e136b8-c8c99b868-b7jb9 to ip-10-0-137-72.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4 to ip-10-0-140-179.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-b-c3bbba-76456ccc86-d86kz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-c3bbba-76456ccc86-d86kz to ip-10-0-137-72.ec2.internal 59m Normal Scheduled pod/payload-processing-e2e-derive-c906b806-6f7d69cd6d-kmczq Successfully assigned openshift-ingress/payload-processing-e2e-derive-c906b806-6f7d69cd6d-kmczq to ip-10-0-137-72.ec2.internal 4m48s Normal Scheduled pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Successfully assigned openshift-ingress/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx to ip-10-0-140-179.ec2.internal 80m Warning FailedScheduling pod/router-default-6789dbc4b5-5gqgl no nodes available to schedule pods 31m Normal Scheduled pod/payload-processing-e2e-ipp-b-c3bbba-55dddc5984-9dxtt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-c3bbba-55dddc5984-9dxtt to ip-10-0-142-3.ec2.internal 31m Normal Scheduled pod/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 Successfully assigned openshift-ingress/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 to ip-10-0-142-3.ec2.internal 67m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 to ip-10-0-137-72.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-inf-a-1eab65-8576d668f6-2lxn5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-1eab65-8576d668f6-2lxn5 to ip-10-0-137-72.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Successfully assigned openshift-ingress/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w to ip-10-0-140-179.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-9zjb7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-9zjb7 to ip-10-0-142-3.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg to ip-10-0-140-179.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h to ip-10-0-137-72.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr Successfully assigned openshift-ingress/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr to ip-10-0-140-179.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-5tsqf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-5tsqf to ip-10-0-137-72.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg to ip-10-0-137-72.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh to ip-10-0-140-179.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-s94wx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-s94wx to ip-10-0-137-72.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5 to ip-10-0-140-179.ec2.internal 59m Normal Scheduled pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n Successfully assigned openshift-ingress/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n to ip-10-0-140-179.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn to ip-10-0-140-179.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc to ip-10-0-137-72.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-dlqvt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-dlqvt to ip-10-0-137-72.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p to ip-10-0-137-72.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-twwlq Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-twwlq to ip-10-0-137-72.ec2.internal 64m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh to ip-10-0-142-3.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm to ip-10-0-137-72.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq to ip-10-0-140-179.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 to ip-10-0-140-179.ec2.internal 49m Normal Scheduled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Successfully assigned openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x to ip-10-0-142-3.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h to ip-10-0-137-72.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-v7qrr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-v7qrr to ip-10-0-137-72.ec2.internal 62m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t to ip-10-0-140-179.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp to ip-10-0-137-72.ec2.internal 51m Normal Scheduled pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj Successfully assigned openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj to ip-10-0-142-3.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 to ip-10-0-142-3.ec2.internal 62m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp to ip-10-0-137-72.ec2.internal 64m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg to ip-10-0-140-179.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf to ip-10-0-140-179.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z to ip-10-0-140-179.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n to ip-10-0-142-3.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d Successfully assigned openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d to ip-10-0-137-72.ec2.internal 60m Normal Scheduled pod/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm Successfully assigned openshift-ingress/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm to ip-10-0-142-3.ec2.internal 72m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-749f59598c-llt9b Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-749f59598c-llt9b to ip-10-0-142-3.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz to ip-10-0-140-179.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Successfully assigned openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk to ip-10-0-140-179.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5 to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g to ip-10-0-137-72.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 to ip-10-0-137-72.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl Successfully assigned openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl to ip-10-0-140-179.ec2.internal 60m Normal Scheduled pod/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f Successfully assigned openshift-ingress/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f to ip-10-0-137-72.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Successfully assigned openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 to ip-10-0-140-179.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2 to ip-10-0-140-179.ec2.internal 64m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g to ip-10-0-137-72.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj to ip-10-0-142-3.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 to ip-10-0-140-179.ec2.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9 to ip-10-0-140-179.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx Successfully assigned openshift-ingress/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx to ip-10-0-142-3.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 to ip-10-0-140-179.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z Successfully assigned openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z to ip-10-0-137-72.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg Successfully assigned openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg to ip-10-0-142-3.ec2.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx to ip-10-0-140-179.ec2.internal 54m Normal Scheduled pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Successfully assigned openshift-ingress/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 to ip-10-0-140-179.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck to ip-10-0-140-179.ec2.internal 67m Normal Scheduled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Successfully assigned openshift-ingress/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn to ip-10-0-137-72.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Successfully assigned openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh to ip-10-0-137-72.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm to ip-10-0-142-3.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6 to ip-10-0-140-179.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj to ip-10-0-142-3.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-mdttb Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-mdttb to ip-10-0-142-3.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg to ip-10-0-140-179.ec2.internal 63m Normal Scheduled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Successfully assigned openshift-ingress/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 to ip-10-0-142-3.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Successfully assigned openshift-ingress/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn to ip-10-0-142-3.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-hm4ww Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-c1da2183-677d9df459-hm4ww to ip-10-0-137-72.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-2t48r Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-2t48r to ip-10-0-137-72.ec2.internal 36m Normal Scheduled pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Successfully assigned openshift-ingress/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz to ip-10-0-140-179.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv to ip-10-0-140-179.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx to ip-10-0-137-72.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc to ip-10-0-140-179.ec2.internal 48m Normal Scheduled pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Successfully assigned openshift-ingress/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 to ip-10-0-140-179.ec2.internal 81m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 81m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 81m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-6789dbc4b5 from 0 to 1 81m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 80m Warning FailedCreate replicaset/router-default-6789dbc4b5 Error creating: pods "router-default-6789dbc4b5-" is forbidden: error fetching namespace "openshift-ingress": unable to find annotation openshift.io/sa.scc.uid-range 80m Normal SuccessfulCreate replicaset/router-default-6789dbc4b5 Created pod: router-default-6789dbc4b5-5gqgl 77m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 76m Warning FailedMount pod/router-default-6789dbc4b5-5gqgl MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 76m Warning FailedMount pod/router-default-6789dbc4b5-5gqgl MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 76m Warning FailedMount pod/router-default-6789dbc4b5-5gqgl MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 76m Warning FailedMount pod/router-default-6789dbc4b5-5gqgl MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 76m Normal Pulled pod/router-default-6789dbc4b5-5gqgl Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 76m Normal Created pod/router-default-6789dbc4b5-5gqgl Container created 76m Normal Started pod/router-default-6789dbc4b5-5gqgl Container started 76m Normal AddedInterface pod/router-default-6789dbc4b5-5gqgl Add eth0 [10.132.0.20/23] from ovn-kubernetes 72m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 72m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 72m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-47kpq 72m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-47kpq Add eth0 [10.133.0.20/23] from ovn-kubernetes 72m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-47kpq Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 72m Normal Started pod/istiod-openshift-gateway-7876bfcb74-47kpq Container started 72m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-47kpq Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.38s (2.38s including waiting). Image size: 185476477 bytes. 72m Normal Created pod/istiod-openshift-gateway-7876bfcb74-47kpq Container created 72m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-47kpq Readiness probe failed: HTTP probe failed with statuscode: 503 72m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-wxstq 72m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 72m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-47kpq Readiness probe error: HTTP probe failed with statuscode: 503... 72m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-749f59598c from 0 to 1 72m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-749f59598c Created pod: openshift-ai-inference-openshift-ai-inference-749f59598c-llt9b 72m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 0 to 1 72m Normal Pulling pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 72m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-749f59598c-llt9b Add eth0 [10.132.0.33/23] from ovn-kubernetes 72m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Add eth0 [10.132.0.32/23] from ovn-kubernetes 72m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-749f59598c-llt9b Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 72m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 72m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-749f59598c-llt9b Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.629s (8.629s including waiting). Image size: 183555333 bytes. 72m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.441s (8.441s including waiting). Image size: 183555333 bytes. 72m Normal Started pod/openshift-ai-inference-openshift-ai-inference-749f59598c-llt9b Container started 72m Normal Created pod/openshift-ai-inference-openshift-ai-inference-749f59598c-llt9b Container created 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-669569fcd5 from 0 to 1 71m Normal SuccessfulCreate replicaset/kube-auth-proxy-5649f7bc66 Created pod: kube-auth-proxy-5649f7bc66-klvr2 71m Normal SuccessfulCreate replicaset/kube-auth-proxy-5649f7bc66 Created pod: kube-auth-proxy-5649f7bc66-trj7m 71m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-669569fcd5 Created pod: data-science-gateway-data-science-gateway-class-669569fcd5lntp7 71m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-5649f7bc66 from 0 to 2 71m Normal Pulling pod/kube-auth-proxy-5649f7bc66-klvr2 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 71m Normal Pulling pod/kube-auth-proxy-5649f7bc66-trj7m Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 71m Normal Started pod/data-science-gateway-data-science-gateway-class-669569fcd5lntp7 Container started 71m Normal Created pod/data-science-gateway-data-science-gateway-class-669569fcd5lntp7 Container created 71m Normal AddedInterface pod/kube-auth-proxy-5649f7bc66-trj7m Add eth0 [10.134.0.26/23] from ovn-kubernetes 71m Normal AddedInterface pod/kube-auth-proxy-5649f7bc66-klvr2 Add eth0 [10.133.0.22/23] from ovn-kubernetes 71m Normal Pulled pod/data-science-gateway-data-science-gateway-class-669569fcd5lntp7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 71m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-669569fcd5lntp7 Add eth0 [10.132.0.42/23] from ovn-kubernetes 71m Normal Started pod/kube-auth-proxy-5649f7bc66-klvr2 Container started 71m Normal Pulled pod/kube-auth-proxy-5649f7bc66-klvr2 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.535s (2.535s including waiting). Image size: 188904677 bytes. 71m Normal Created pod/kube-auth-proxy-5649f7bc66-klvr2 Container created 71m Normal Created pod/kube-auth-proxy-5649f7bc66-trj7m Container created 71m Normal Started pod/kube-auth-proxy-5649f7bc66-trj7m Container started 71m Normal Pulled pod/kube-auth-proxy-5649f7bc66-trj7m Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.84s (2.84s including waiting). Image size: 188904677 bytes. 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 70m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-jggkz 70m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 70m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-jggkz Container created 70m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-jggkz Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 70m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-jggkz Container started 70m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-jggkz Add eth0 [10.133.0.25/23] from ovn-kubernetes 70m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-47kpq 70m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-47kpq Stopping container discovery 70m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 70m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 70m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 69m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Container created 69m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 69m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Container started 69m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Startup probe failed: Get "http://10.132.0.32:15021/healthz/ready": dial tcp 10.132.0.32:15021: connect: connection refused 69m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Startup probe error: Get "http://10.132.0.32:15021/healthz/ready": dial tcp 10.132.0.32:15021: connect: connection refused... 69m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 69m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Add eth0 [10.134.0.42/23] from ovn-kubernetes 68m Normal Pulling pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 68m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 1 to 4 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-p8tnk 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-lkltg 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-2ff29 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Add eth0 [10.134.0.41/23] from ovn-kubernetes 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Add eth0 [10.133.0.36/23] from ovn-kubernetes 68m Normal Pulling pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 68m Normal Pulling pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 68m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Container started 68m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Container created 68m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.675s (2.675s including waiting). Image size: 183555333 bytes. 68m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Container created 68m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Container started 68m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.733s (3.733s including waiting). Image size: 183555333 bytes. 68m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Container created 68m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Container started 68m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.765s (3.765s including waiting). Image size: 183555333 bytes. 67m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-tbwcl 67m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 67m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 67m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-kjg7p 67m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 67m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 67m Normal Pulling pod/payload-pre-processing-7b98bb678c-tbwcl Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 67m Normal Pulling pod/payload-processing-5f78687d5b-kjg7p Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 67m Normal AddedInterface pod/payload-processing-5f78687d5b-kjg7p Add eth0 [10.133.0.38/23] from ovn-kubernetes 67m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-tbwcl Add eth0 [10.133.0.37/23] from ovn-kubernetes 67m Normal Created pod/payload-pre-processing-7b98bb678c-tbwcl Container created 67m Normal Started pod/payload-pre-processing-7b98bb678c-tbwcl Container started 67m Normal Started pod/payload-processing-5f78687d5b-kjg7p Container started 67m Normal Pulled pod/payload-pre-processing-7b98bb678c-tbwcl Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.367s (3.367s including waiting). Image size: 125353965 bytes. 67m Normal Created pod/payload-processing-5f78687d5b-kjg7p Container created 67m Normal Pulled pod/payload-processing-5f78687d5b-kjg7p Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.298s (3.298s including waiting). Image size: 125353965 bytes. 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused 67m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 67m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 67m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 67m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 67m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 67m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 67m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Add eth0 [10.133.0.39/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 4 to 6 67m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 67m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-cdm7z 67m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Add eth0 [10.132.0.48/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-4nxfb 67m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Container created 67m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Container started 67m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Container started 67m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Container created 67m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused 67m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused... 67m Normal AddedLabel gateway/e2e-worker-w3-79477e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-79477e 67m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq 67m Normal AddedLabel gateway/e2e-worker-w2-2cda44 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-2cda44 67m Normal AddedLabel gateway/e2e-worker-w5-43c070 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-43c070 67m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled up replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 0 to 1 67m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 67m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn 67m Normal AddedLabel gateway/e2e-worker-w4-839163 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-839163 67m Normal AddedLabel gateway/e2e-worker-w0-58fea9 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-58fea9 67m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled up replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 0 to 1 67m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled up replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 0 to 1 67m Normal AddedLabel gateway/e2e-worker-w1-56ac74 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-56ac74 67m Normal AddedLabel gateway/e2e-worker-w6-ed088d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-ed088d 67m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled up replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 0 to 1 67m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 67m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz 67m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled up replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 0 to 1 67m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz 67m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Add eth0 [10.134.0.45/23] from ovn-kubernetes 67m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Add eth0 [10.134.0.44/23] from ovn-kubernetes 67m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Add eth0 [10.133.0.42/23] from ovn-kubernetes 67m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Add eth0 [10.133.0.40/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z 67m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz Add eth0 [10.133.0.41/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled up replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 0 to 1 67m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 Add eth0 [10.134.0.46/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled up replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 0 to 1 67m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Add eth0 [10.134.0.47/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-56ac74 Scaled up replica set payload-processing-e2e-worker-w1-56ac74-96bdd884 from 0 to 1 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-56ac74-96bdd884 Created pod: payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-56ac74-96bdd884 Error creating: pods "payload-processing-e2e-worker-w1-56ac74-96bdd884-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-56ac74: serviceaccount "payload-processing-e2e-worker-w1-56ac74" not found 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767 Error creating: pods "payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-56ac74: serviceaccount "payload-processing-e2e-worker-w1-56ac74" not found 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767 Created pod: payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-56ac74 Scaled up replica set payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767 from 0 to 1 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs Add eth0 [10.133.0.43/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-79477e Scaled up replica set payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4 from 0 to 1 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-79477e Scaled up replica set payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c from 0 to 1 67m Normal Started pod/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs Container started 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4 Error creating: pods "payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-79477e: serviceaccount "payload-processing-e2e-worker-w3-79477e" not found 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c Error creating: pods "payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-79477e: serviceaccount "payload-processing-e2e-worker-w3-79477e" not found 67m Normal Created pod/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs Container created 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc Add eth0 [10.134.0.48/23] from ovn-kubernetes 67m Normal Pulling pod/payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9 Error creating: pods "payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-58fea9: serviceaccount "payload-processing-e2e-worker-w0-58fea9" not found 66m Normal Created pod/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 Container created 66m Normal Started pod/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z Container started 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-58fea9 Scaled up replica set payload-processing-e2e-worker-w0-58fea9-9984455b6 from 0 to 1 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9 Created pod: payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c Created pod: payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z 66m Normal Started pod/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 Container started 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4 Created pod: payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 66m Normal Pulled pod/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-58fea9-9984455b6 Error creating: pods "payload-processing-e2e-worker-w0-58fea9-9984455b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-58fea9: serviceaccount "payload-processing-e2e-worker-w0-58fea9" not found 66m Normal AddedInterface pod/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 Add eth0 [10.133.0.45/23] from ovn-kubernetes 66m Normal Created pod/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z Container created 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z Add eth0 [10.133.0.44/23] from ovn-kubernetes 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-58fea9 Scaled up replica set payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9 from 0 to 1 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-58fea9-9984455b6 Created pod: payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 66m Normal Created pod/payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 Container created 66m Normal Created pod/payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc Container created 66m Normal Started pod/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db Container started 66m Normal Started pod/payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 Container started 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db Add eth0 [10.133.0.47/23] from ovn-kubernetes 66m Normal AddedInterface pod/payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 Add eth0 [10.134.0.49/23] from ovn-kubernetes 66m Normal Pulled pod/payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Created pod/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db Container created 66m Normal Pulled pod/payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.008s (2.008s including waiting). Image size: 125353965 bytes. 66m Normal Started pod/payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc Container started 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-43c070 Scaled up replica set payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579 from 0 to 1 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-43c070 Scaled up replica set payload-processing-e2e-worker-w5-43c070-78859bb9c8 from 0 to 1 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-43c070-78859bb9c8 Error creating: pods "payload-processing-e2e-worker-w5-43c070-78859bb9c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-43c070: serviceaccount "payload-processing-e2e-worker-w5-43c070" not found 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579 Error creating: pods "payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-43c070: serviceaccount "payload-processing-e2e-worker-w5-43c070" not found 66m Normal Created pod/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j Container created 66m Normal AddedInterface pod/payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 Add eth0 [10.134.0.50/23] from ovn-kubernetes 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Started pod/payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 Container started 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j Add eth0 [10.134.0.51/23] from ovn-kubernetes 66m Normal Created pod/payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 Container created 66m Normal Pulled pod/payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-43c070-78859bb9c8 Created pod: payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-ed088d-dfb468776 Created pod: payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h 66m Normal Started pod/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j Container started 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-ed088d-dfb468776 Error creating: pods "payload-processing-e2e-worker-w6-ed088d-dfb468776-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-ed088d: serviceaccount "payload-processing-e2e-worker-w6-ed088d" not found 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955 Created pod: payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-ed088d Scaled up replica set payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955 from 0 to 1 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579 Created pod: payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-ed088d Scaled up replica set payload-processing-e2e-worker-w6-ed088d-dfb468776 from 0 to 1 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955 Error creating: pods "payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-ed088d: serviceaccount "payload-processing-e2e-worker-w6-ed088d" not found 66m Normal Started pod/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j Container started 66m Normal AddedInterface pod/payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h Add eth0 [10.134.0.52/23] from ovn-kubernetes 66m Normal Created pod/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j Container created 66m Normal Pulled pod/payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Created pod/payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h Container created 66m Normal Started pod/payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h Container started 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j Add eth0 [10.134.0.53/23] from ovn-kubernetes 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4 Created pod: payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm 66m Normal AddedInterface pod/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm Add eth0 [10.132.0.49/23] from ovn-kubernetes 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq Add eth0 [10.134.0.54/23] from ovn-kubernetes 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-839163 Scaled up replica set payload-processing-e2e-worker-w4-839163-5bf99f9954 from 0 to 1 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf Error creating: pods "payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-2cda44: serviceaccount "payload-processing-e2e-worker-w2-2cda44" not found 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf Created pod: payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-2cda44 Scaled up replica set payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf from 0 to 1 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-2cda44 Scaled up replica set payload-processing-e2e-worker-w2-2cda44-76f8dccbf4 from 0 to 1 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4 Error creating: pods "payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-2cda44: serviceaccount "payload-processing-e2e-worker-w2-2cda44" not found 66m Normal Pulling pod/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal AddedInterface pod/payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 Add eth0 [10.132.0.50/23] from ovn-kubernetes 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft Add eth0 [10.132.0.51/23] from ovn-kubernetes 66m Normal Created pod/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq Container created 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c Created pod: payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-839163 Scaled up replica set payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c from 0 to 1 66m Normal Started pod/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq Container started 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-839163-5bf99f9954 Created pod: payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 66m Normal Pulling pod/payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c Error creating: pods "payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-839163: serviceaccount "payload-processing-e2e-worker-w4-839163" not found 66m Normal Pulling pod/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-839163-5bf99f9954 Error creating: pods "payload-processing-e2e-worker-w4-839163-5bf99f9954-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-839163: serviceaccount "payload-processing-e2e-worker-w4-839163" not found 66m Normal Pulled pod/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.773s (2.773s including waiting). Image size: 125353965 bytes. 66m Normal Created pod/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm Container created 66m Normal Started pod/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm Container started 66m Normal Started pod/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft Container started 66m Normal Created pod/payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 Container created 66m Normal Created pod/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft Container created 66m Normal Pulled pod/payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.454s (2.454s including waiting). Image size: 125353965 bytes. 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.406s (2.406s including waiting). Image size: 125353965 bytes. 66m Normal Started pod/payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 Container started 66m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz Container created 66m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz Container started 66m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Container created 66m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Container started 66m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused 66m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Container started 66m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Container created 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Container started 66m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Container created 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 Container created 66m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 Container started 66m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Readiness probe error: Get "http://10.133.0.42:15021/healthz/ready": read tcp 10.133.0.2:41194->10.133.0.42:15021: read: connection reset by peer... 66m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Readiness probe failed: Get "http://10.134.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 66m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Readiness probe error: Get "http://10.134.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 66m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Readiness probe failed: Get "http://10.133.0.42:15021/healthz/ready": read tcp 10.133.0.2:41194->10.133.0.42:15021: read: connection reset by peer 66m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Container started 66m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Container created 66m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused... 66m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Container started 66m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Container created 66m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused... 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled up replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 1 to 2 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Add eth0 [10.133.0.53/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj 65m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 65m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled up replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 1 to 2 65m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Container started 65m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Container created 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Container created 65m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Container started 65m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Add eth0 [10.133.0.54/23] from ovn-kubernetes 65m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 64m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 64m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Container started 64m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled up replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 1 to 2 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg 64m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled up replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 1 to 2 64m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Add eth0 [10.133.0.55/23] from ovn-kubernetes 64m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled up replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 1 to 2 64m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Container created 64m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Container created 64m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Container created 64m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Add eth0 [10.134.0.62/23] from ovn-kubernetes 64m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Add eth0 [10.133.0.56/23] from ovn-kubernetes 64m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Container started 64m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Container started 64m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 64m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 64m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 64m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 64m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled up replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 1 to 2 64m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Container started 64m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Container created 64m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Add eth0 [10.134.0.71/23] from ovn-kubernetes 64m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 64m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 64m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 64m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g 64m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh 64m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled up replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 2 to 4 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v 64m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 64m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled up replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 2 to 4 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 64m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Add eth0 [10.132.0.67/23] from ovn-kubernetes 64m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Container created 64m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Add eth0 [10.134.0.75/23] from ovn-kubernetes 64m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Container created 64m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Add eth0 [10.133.0.57/23] from ovn-kubernetes 64m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Container started 64m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Container started 64m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Container started 64m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Container started 64m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Container created 64m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Add eth0 [10.132.0.68/23] from ovn-kubernetes 64m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Container created 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-2tsvw 64m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 6 to 9 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-vp9n6 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-m9l2r 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Add eth0 [10.132.0.70/23] from ovn-kubernetes 64m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 Container started 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Add eth0 [10.133.0.58/23] from ovn-kubernetes 64m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Container created 64m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Container started 64m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Container started 64m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Container created 64m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 Container created 64m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 Add eth0 [10.134.0.76/23] from ovn-kubernetes 63m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 63m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 63m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 63m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 63m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused... 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled up replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 2 to 4 63m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w Add eth0 [10.134.0.78/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw Container started 63m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w Container created 63m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w Container started 63m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw Container created 63m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw Add eth0 [10.132.0.71/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg 63m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled up replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 2 to 4 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w 63m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v 63m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Container created 63m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Container started 63m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Container started 63m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Container created 63m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Add eth0 [10.132.0.72/23] from ovn-kubernetes 63m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Add eth0 [10.133.0.59/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 (combined from similar events): Created pod: maas-default-gateway-openshift-default-57f78bcd55-7kdrm 63m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-7kdrm Container created 63m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-7kdrm Container started 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-7kdrm Add eth0 [10.134.0.81/23] from ovn-kubernetes 63m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-7kdrm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-7kdrm Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 63m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-7kdrm Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck 63m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled up replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 1 to 2 63m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n 63m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled up replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 2 to 4 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn Add eth0 [10.133.0.62/23] from ovn-kubernetes 63m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n Container created 63m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Add eth0 [10.134.0.82/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm 63m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Add eth0 [10.133.0.61/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n Container started 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Container created 63m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled up replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 2 to 4 63m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n Add eth0 [10.132.0.75/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Container started 63m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Container created 63m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Container started 63m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Add eth0 [10.132.0.76/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Container created 63m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Container started 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn 63m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn Container created 63m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn Container started 63m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g 63m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled up replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 4 to 6 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 6; reason: 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 63m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Container created 63m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled up replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 4 to 5 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 5; reason: 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn 63m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Add eth0 [10.134.0.85/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn Container started 63m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn Container created 63m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn Add eth0 [10.134.0.86/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Container started 63m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Add eth0 [10.132.0.77/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Container started 63m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Container created 63m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 63m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled up replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 4 to 6 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm 63m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Add eth0 [10.133.0.64/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Container created 63m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Container started 63m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Container created 63m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Container started 63m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Add eth0 [10.132.0.79/23] from ovn-kubernetes 63m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 62m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled up replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 6 to 8 62m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled up replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 4 to 8 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-m595f 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t 62m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp 62m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql Container started 62m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Container created 62m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t Add eth0 [10.133.0.66/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t Container created 62m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t Container started 62m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Container started 62m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp Container started 62m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp Container created 62m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp Add eth0 [10.134.0.90/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Container started 62m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Add eth0 [10.132.0.81/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Container created 62m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Add eth0 [10.133.0.67/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Add eth0 [10.134.0.91/23] from ovn-kubernetes 62m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Container created 62m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Container started 62m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql Container created 62m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql Add eth0 [10.134.0.92/23] from ovn-kubernetes 62m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 62m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled up replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 4 to 7 62m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Container started 62m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Add eth0 [10.132.0.82/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 Add eth0 [10.133.0.69/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Container started 62m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Container created 62m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Add eth0 [10.133.0.70/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Container created 62m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 Container created 62m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 Container started 62m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 (combined from similar events): Created pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 62m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled up replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 6 to 10 62m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw Container started 62m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Container started 62m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 Add eth0 [10.134.0.96/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp Container started 62m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp Container created 62m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 Container created 62m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp Add eth0 [10.134.0.95/23] from ovn-kubernetes 62m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw Container created 62m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Add eth0 [10.133.0.71/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Container created 62m Normal Started pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 Container started 62m Normal Pulled pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw Add eth0 [10.132.0.85/23] from ovn-kubernetes 61m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 61m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled up replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 8 to 9 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 61m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 Container started 61m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 Container created 61m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 Add eth0 [10.132.0.92/23] from ovn-kubernetes 61m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Readiness probe error: Get "http://10.133.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Readiness probe failed: Get "http://10.133.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled up replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 4 to 7 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled up replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 2 to 4 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 61m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf Container created 61m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw Container started 61m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 Add eth0 [10.132.0.95/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled up replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 5 to 10 61m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg Container started 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 61m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw Container created 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 61m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 Add eth0 [10.133.0.72/23] from ovn-kubernetes 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 Container created 61m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 Container started 61m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 Container created 61m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 Container started 61m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg Container created 61m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf Add eth0 [10.132.0.94/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 (combined from similar events): Created pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 61m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf Container started 61m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg Add eth0 [10.134.0.106/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw Add eth0 [10.134.0.105/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Container created 61m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Add eth0 [10.132.0.96/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 Container started 61m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 Container created 61m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Container created 61m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Container started 61m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Container started 61m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Add eth0 [10.134.0.107/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 Container started 61m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 Add eth0 [10.133.0.74/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 Container created 61m Normal Pulled pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Add eth0 [10.132.0.97/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 Add eth0 [10.133.0.73/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Container started 61m Normal Created pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Container created 61m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 61m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled up replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 9 to 10 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled up replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 4 to 5 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 5; reason: 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 (combined from similar events): Created pod: e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq 61m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Container started 61m Normal Pulled pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Container started 61m Normal AddedInterface pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Add eth0 [10.133.0.75/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Container created 61m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Container created 61m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Add eth0 [10.134.0.111/23] from ovn-kubernetes 61m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Readiness probe error: Get "http://10.134.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Readiness probe failed: Get "http://10.134.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df (combined from similar events): Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Created pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 (combined from similar events): Created pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled up replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 8 to 10 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled up replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 7 to 10 61m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 Add eth0 [10.132.0.101/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 Add eth0 [10.132.0.102/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm Container created 61m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk Container created 61m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 Container created 61m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 Container started 61m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm Container started 61m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk Add eth0 [10.134.0.115/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 Container started 61m Normal Created pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 Container created 61m Normal Pulled pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk Container started 61m Normal AddedInterface pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm Add eth0 [10.134.0.114/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf Container started 61m Normal Created pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf Container created 61m Normal Pulled pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf Add eth0 [10.133.0.76/23] from ovn-kubernetes 60m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm Startup probe error: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm Startup probe failed: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal AddedLabel gateway/e2e-migrate-dd1e164d-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-dd1e164d-gw 60m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Readiness probe failed: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Normal SuccessfulCreate replicaset/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9 Created pod: e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f 60m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Readiness probe failed: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Readiness probe error: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal ScalingReplicaSet deployment/e2e-migrate-dd1e164d-gw-openshift-default Scaled up replica set e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9 from 0 to 1 60m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Readiness probe error: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal Created pod/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f Container created 60m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Readiness probe failed: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Readiness probe error: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Readiness probe error: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Readiness probe failed: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Normal AddedInterface pod/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f Add eth0 [10.134.0.120/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f Container started 60m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Normal AddedLabel gateway/e2e-ait-92d5d744 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-92d5d744 60m Normal AddedLabel gateway/e2e-ait-4b62455c Added label istio.io/rev=openshift-gateway to gateway e2e-ait-4b62455c 60m Normal ScalingReplicaSet deployment/e2e-ait-92d5d744-openshift-default Scaled up replica set e2e-ait-92d5d744-openshift-default-54d7f568d7 from 0 to 1 60m Normal SuccessfulCreate replicaset/e2e-ait-92d5d744-openshift-default-54d7f568d7 Created pod: e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh 60m Normal Started pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh Container started 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-dd1e164d Scaled up replica set payload-processing-e2e-migrate-dd1e164d-5d78f9cfd from 0 to 1 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-dd1e164d Scaled up replica set payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c from 0 to 1 60m Normal AddedInterface pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh Add eth0 [10.133.0.78/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh Container created 60m Normal ScalingReplicaSet deployment/e2e-ait-4b62455c-openshift-default Scaled up replica set e2e-ait-4b62455c-openshift-default-54ff4fcf5 from 0 to 1 60m Warning FailedCreate replicaset/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd Error creating: pods "payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-dd1e164d: serviceaccount "payload-processing-e2e-migrate-dd1e164d" not found 60m Normal SuccessfulCreate replicaset/e2e-ait-4b62455c-openshift-default-54ff4fcf5 Created pod: e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd Created pod: payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-h7dcj 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c Error creating: pods "payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-dd1e164d: serviceaccount "payload-processing-e2e-migrate-dd1e164d" not found 60m Normal Started pod/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-h7dcj Container started 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c Created pod: payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9 60m Normal Created pod/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-h7dcj Container created 60m Normal Pulled pod/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-h7dcj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-h7dcj Add eth0 [10.134.0.121/23] from ovn-kubernetes 60m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m 60m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c (combined from similar events): Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 60m Normal SuccessfulCreate replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Created pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp 60m Normal Started pod/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9 Container started 60m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9 Add eth0 [10.133.0.80/23] from ovn-kubernetes 60m Normal Pulled pod/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled up replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 7 to 10 60m Normal Started pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz Container started 60m Normal Created pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz Container created 60m Normal Pulled pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz Add eth0 [10.133.0.79/23] from ovn-kubernetes 60m Normal Created pod/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9 Container created 60m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp Add eth0 [10.132.0.107/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 Add eth0 [10.134.0.123/23] from ovn-kubernetes 60m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m Container created 60m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 Container created 60m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 Container started 60m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m Container started 60m Normal AddedInterface pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m Add eth0 [10.133.0.81/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp Container started 60m Normal Pulled pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp Container created 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-92d5d744 Scaled up replica set payload-processing-e2e-ait-92d5d744-779597976 from 0 to 1 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-92d5d744-5c548bc679 Error creating: pods "payload-pre-processing-e2e-ait-92d5d744-5c548bc679-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-92d5d744: serviceaccount "payload-processing-e2e-ait-92d5d744" not found 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-92d5d744-5c548bc679 Created pod: payload-pre-processing-e2e-ait-92d5d744-5c548bc679-rtwfr 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-92d5d744 Scaled up replica set payload-pre-processing-e2e-ait-92d5d744-5c548bc679 from 0 to 1 60m Warning FailedCreate replicaset/payload-processing-e2e-ait-92d5d744-779597976 Error creating: pods "payload-processing-e2e-ait-92d5d744-779597976-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-92d5d744: serviceaccount "payload-processing-e2e-ait-92d5d744" not found 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-92d5d744-779597976 Created pod: payload-processing-e2e-ait-92d5d744-779597976-7xjvh 60m Normal Pulled pod/payload-processing-e2e-ait-92d5d744-779597976-7xjvh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Started pod/payload-pre-processing-e2e-ait-92d5d744-5c548bc679-rtwfr Container started 60m Normal Created pod/payload-processing-e2e-ait-92d5d744-779597976-7xjvh Container created 60m Normal AddedInterface pod/payload-processing-e2e-ait-92d5d744-779597976-7xjvh Add eth0 [10.133.0.83/23] from ovn-kubernetes 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-4b62455c Scaled up replica set payload-pre-processing-e2e-ait-4b62455c-7b9674c54c from 0 to 1 60m Normal Created pod/payload-pre-processing-e2e-ait-92d5d744-5c548bc679-rtwfr Container created 60m Normal Pulled pod/payload-pre-processing-e2e-ait-92d5d744-5c548bc679-rtwfr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/payload-pre-processing-e2e-ait-92d5d744-5c548bc679-rtwfr Add eth0 [10.134.0.124/23] from ovn-kubernetes 60m Normal Started pod/payload-processing-e2e-ait-92d5d744-779597976-7xjvh Container started 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-4b62455c Scaled up replica set payload-processing-e2e-ait-4b62455c-5945779696 from 0 to 1 60m Normal AddedInterface pod/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-mf2db Add eth0 [10.134.0.125/23] from ovn-kubernetes 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c Error creating: pods "payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4b62455c: serviceaccount "payload-processing-e2e-ait-4b62455c" not found 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-4b62455c-5945779696 Created pod: payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 60m Warning FailedCreate replicaset/payload-processing-e2e-ait-4b62455c-5945779696 Error creating: pods "payload-processing-e2e-ait-4b62455c-5945779696-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4b62455c: serviceaccount "payload-processing-e2e-ait-4b62455c" not found 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c Created pod: payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-mf2db 60m Normal Pulled pod/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-mf2db Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Started pod/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-mf2db Container started 60m Normal Created pod/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 Container created 60m Normal AddedInterface pod/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 Add eth0 [10.133.0.84/23] from ovn-kubernetes 60m Normal Started pod/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 Container started 60m Normal Created pod/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-mf2db Container created 60m Normal Pulled pod/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 Startup probe failed: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 Startup probe error: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-4b62455c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-4b62455c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-dtvjx 60m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Add eth0 [10.134.0.128/23] from ovn-kubernetes 60m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled up replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 5 to 10 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f (combined from similar events): Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Created pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr 60m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Container created 60m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Container started 60m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Add eth0 [10.133.0.85/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Container created 60m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Container started 60m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Container created 60m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Add eth0 [10.132.0.108/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Add eth0 [10.132.0.109/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Container started 60m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 Container started 60m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Container created 60m Normal Created pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 Container created 60m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Pulled pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 Add eth0 [10.134.0.129/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Add eth0 [10.133.0.86/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Container started 60m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.644s (3.644s including waiting). Image size: 185476477 bytes. 60m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Container created 60m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Container started 60m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Readiness probe failed: HTTP probe failed with statuscode: 503 60m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Readiness probe error: HTTP probe failed with statuscode: 503... 60m Normal ScalingReplicaSet deployment/e2e-mt-1f954ca9-openshift-default Scaled up replica set e2e-mt-1f954ca9-openshift-default-ffd455d85 from 0 to 1 60m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 Startup probe error: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 Startup probe failed: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused 60m Normal AddedLabel gateway/e2e-mt-1f954ca9 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-1f954ca9 60m Normal SuccessfulCreate replicaset/e2e-mt-1f954ca9-openshift-default-ffd455d85 Created pod: e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-dd1e164d-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Normal AddedInterface pod/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm Add eth0 [10.132.0.112/23] from ovn-kubernetes 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-dd1e164d-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Readiness probe failed: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Readiness probe error: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Readiness probe error: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Readiness probe failed: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Readiness probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Readiness probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Readiness probe error: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Readiness probe failed: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-4b62455c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-4b62455c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 60m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-8s4mz 60m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 60m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Readiness probe failed: Get "http://10.132.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Readiness probe error: Get "http://10.132.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Add eth0 [10.132.0.114/23] from ovn-kubernetes 60m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9 Error creating: pods "payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-1f954ca9: serviceaccount "payload-processing-e2e-mt-1f954ca9" not found 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-1f954ca9 Scaled up replica set payload-pre-processing-e2e-mt-1f954ca9-549d445bc9 from 0 to 1 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-1f954ca9 Scaled up replica set payload-processing-e2e-mt-1f954ca9-5d46f4f796 from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-1f954ca9-5d46f4f796 Created pod: payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w 60m Warning FailedCreate replicaset/payload-processing-e2e-mt-1f954ca9-5d46f4f796 Error creating: pods "payload-processing-e2e-mt-1f954ca9-5d46f4f796-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-1f954ca9: serviceaccount "payload-processing-e2e-mt-1f954ca9" not found 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9 Created pod: payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx 59m Normal Started pod/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w Container started 59m Normal Pulled pod/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w Container created 59m Normal Created pod/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx Container created 59m Normal AddedInterface pod/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w Add eth0 [10.133.0.88/23] from ovn-kubernetes 59m Normal AddedInterface pod/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx Add eth0 [10.133.0.89/23] from ovn-kubernetes 59m Normal Started pod/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx Container started 59m Normal Pulled pod/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.143s (3.143s including waiting). Image size: 185476477 bytes. 59m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Container started 59m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Container created 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-92d5d744-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-92d5d744-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal Killing pod/payload-processing-e2e-migrate-dd1e164d-5d78f9cfd-h7dcj Stopping container payload-processing 59m Normal Killing pod/payload-pre-processing-e2e-migrate-dd1e164d-b6d759b8c-mrdk9 Stopping container payload-pre-processing 59m Normal Killing pod/payload-processing-e2e-ait-92d5d744-779597976-7xjvh Stopping container payload-processing 59m Normal Killing pod/payload-pre-processing-e2e-ait-92d5d744-5c548bc679-rtwfr Stopping container payload-pre-processing 59m Normal Killing pod/payload-pre-processing-e2e-ait-4b62455c-7b9674c54c-mf2db Stopping container payload-pre-processing 59m Normal Killing pod/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f Stopping container istio-proxy 59m Normal AddedLabel gateway/e2e-ait-399049f3 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-399049f3 59m Normal SuccessfulCreate replicaset/e2e-ait-399049f3-openshift-default-778658c8c Created pod: e2e-ait-399049f3-openshift-default-778658c8c-ltmlv 59m Normal Killing pod/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 Stopping container payload-processing 59m Warning Unhealthy pod/payload-processing-e2e-ait-4b62455c-5945779696-2zmk7 Readiness probe failed: timeout: health rpc did not complete within 1s 59m Normal ScalingReplicaSet deployment/e2e-ait-399049f3-openshift-default Scaled up replica set e2e-ait-399049f3-openshift-default-778658c8c from 0 to 1 59m Normal AddedInterface pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Add eth0 [10.133.0.91/23] from ovn-kubernetes 59m Warning ProbeError pod/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f Readiness probe error: Get "http://10.134.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Normal Pulled pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning Unhealthy pod/e2e-migrate-dd1e164d-gw-openshift-default-86c9fdcd9-fmz9f Readiness probe failed: Get "http://10.134.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal ScalingReplicaSet deployment/e2e-ait-92d5d744-openshift-default Scaled up replica set e2e-ait-92d5d744-openshift-default-54d7f568d7 from 0 to 1 59m Warning FailedCreate replicaset/e2e-ait-92d5d744-openshift-default-54d7f568d7 Error creating: pods "e2e-ait-92d5d744-openshift-default-54d7f568d7-" is forbidden: error looking up service account openshift-ingress/e2e-ait-92d5d744-openshift-default: serviceaccount "e2e-ait-92d5d744-openshift-default" not found 59m Normal Created pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Container created 59m Normal Started pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Container started 59m Normal Killing pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh Stopping container istio-proxy 59m Normal SuccessfulCreate replicaset/e2e-derive-c906b806-openshift-default-6c76dc5969 Created pod: e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n 59m Normal AddedLabel gateway/e2e-derive-c906b806 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-c906b806 59m Warning ProbeError pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Startup probe error: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Startup probe failed: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused 59m Normal ScalingReplicaSet deployment/e2e-derive-c906b806-openshift-default Scaled up replica set e2e-derive-c906b806-openshift-default-6c76dc5969 from 0 to 1 59m Normal Pulled pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n Add eth0 [10.133.0.92/23] from ovn-kubernetes 59m Normal Killing pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz Stopping container istio-proxy 59m Warning ProbeError pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh Readiness probe error: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-ait-92d5d744-openshift-default-54d7f568d7-2xkdh Readiness probe failed: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal Started pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n Container started 59m Normal Created pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n Container created 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-399049f3 Scaled up replica set payload-processing-e2e-ait-399049f3-644865fb9b from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-399049f3-644865fb9b Created pod: payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-399049f3-77f856b49f Created pod: payload-pre-processing-e2e-ait-399049f3-77f856b49f-f24sl 59m Warning FailedCreate replicaset/payload-processing-e2e-ait-399049f3-644865fb9b Error creating: pods "payload-processing-e2e-ait-399049f3-644865fb9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-399049f3: serviceaccount "payload-processing-e2e-ait-399049f3" not found 59m Warning Unhealthy pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz Readiness probe failed: Get "http://10.133.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-399049f3 Scaled up replica set payload-pre-processing-e2e-ait-399049f3-77f856b49f from 0 to 1 59m Warning ProbeError pod/e2e-ait-4b62455c-openshift-default-54ff4fcf5-575pz Readiness probe error: Get "http://10.133.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-399049f3-77f856b49f Error creating: pods "payload-pre-processing-e2e-ait-399049f3-77f856b49f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-399049f3: serviceaccount "payload-processing-e2e-ait-399049f3" not found 59m Normal Created pod/payload-pre-processing-e2e-ait-399049f3-77f856b49f-f24sl Container created 59m Normal Started pod/payload-pre-processing-e2e-ait-399049f3-77f856b49f-f24sl Container started 59m Normal Created pod/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h Container created 59m Normal Started pod/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h Container started 59m Normal Pulled pod/payload-pre-processing-e2e-ait-399049f3-77f856b49f-f24sl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h Add eth0 [10.133.0.93/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-pre-processing-e2e-ait-399049f3-77f856b49f-f24sl Add eth0 [10.134.0.132/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-c906b806 Scaled up replica set payload-processing-e2e-derive-c906b806-6f7d69cd6d from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-c906b806-9477cb87f Error creating: pods "payload-pre-processing-e2e-derive-c906b806-9477cb87f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c906b806: serviceaccount "payload-processing-e2e-derive-c906b806" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-c906b806 Scaled up replica set payload-pre-processing-e2e-derive-c906b806-9477cb87f from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-c906b806-9477cb87f Created pod: payload-pre-processing-e2e-derive-c906b806-9477cb87f-bgv72 59m Warning FailedCreate replicaset/payload-processing-e2e-derive-c906b806-6f7d69cd6d Error creating: pods "payload-processing-e2e-derive-c906b806-6f7d69cd6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c906b806: serviceaccount "payload-processing-e2e-derive-c906b806" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-c906b806-6f7d69cd6d Created pod: payload-processing-e2e-derive-c906b806-6f7d69cd6d-kmczq 59m Normal Started pod/payload-processing-e2e-derive-c906b806-6f7d69cd6d-kmczq Container started 59m Normal AddedInterface pod/payload-processing-e2e-derive-c906b806-6f7d69cd6d-kmczq Add eth0 [10.134.0.133/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-derive-c906b806-6f7d69cd6d-kmczq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-processing-e2e-derive-c906b806-6f7d69cd6d-kmczq Container created 59m Normal AddedInterface pod/payload-pre-processing-e2e-derive-c906b806-9477cb87f-bgv72 Add eth0 [10.134.0.134/23] from ovn-kubernetes 59m Normal Pulled pod/payload-pre-processing-e2e-derive-c906b806-9477cb87f-bgv72 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Started pod/payload-pre-processing-e2e-derive-c906b806-9477cb87f-bgv72 Container started 59m Normal Created pod/payload-pre-processing-e2e-derive-c906b806-9477cb87f-bgv72 Container created 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-399049f3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-399049f3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-c906b806-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-c906b806-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal Pulled pod/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm Container created 59m Normal Started pod/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm Container started 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-1f954ca9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-1f954ca9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-399049f3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-399049f3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning Unhealthy pod/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm Startup probe failed: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused 58m Warning ProbeError pod/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused... 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-c906b806-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-c906b806-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Normal Killing pod/payload-processing-e2e-ait-399049f3-644865fb9b-2xv4h Stopping container payload-processing 58m Normal Killing pod/payload-pre-processing-e2e-ait-399049f3-77f856b49f-f24sl Stopping container payload-pre-processing 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-1f954ca9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-1f954ca9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal Killing pod/payload-pre-processing-e2e-derive-c906b806-9477cb87f-bgv72 Stopping container payload-pre-processing 58m Normal Killing pod/payload-processing-e2e-derive-c906b806-6f7d69cd6d-kmczq Stopping container payload-processing 58m Normal Killing pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Stopping container istio-proxy 58m Normal Killing pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n Stopping container istio-proxy 58m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e11bef-openshift-default Scaled up replica set e2e-shared-a-w4-e11bef-openshift-default-689d85bc75 from 0 to 1 58m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75 Created pod: e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq 58m Normal AddedLabel gateway/e2e-shared-a-w4-e11bef Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-e11bef 58m Normal AddedInterface pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Add eth0 [10.133.0.96/23] from ovn-kubernetes 58m Warning ProbeError pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Readiness probe error: Get "http://10.133.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-ait-399049f3-openshift-default-778658c8c-ltmlv Readiness probe failed: Get "http://10.133.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n Readiness probe error: Get "http://10.133.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-derive-c906b806-openshift-default-6c76dc5969-ng42n Readiness probe failed: Get "http://10.133.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-e11bef Scaled up replica set payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-e11bef Scaled up replica set payload-processing-e2e-shared-a-w4-e11bef-847d99c87b from 0 to 1 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c Error creating: pods "payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef: serviceaccount "payload-processing-e2e-shared-a-w4-e11bef" not found 58m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b Error creating: pods "payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-e11bef: serviceaccount "payload-processing-e2e-shared-a-w4-e11bef" not found 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c Created pod: payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 58m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 Add eth0 [10.133.0.97/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b Created pod: payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm 58m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm Add eth0 [10.133.0.98/23] from ovn-kubernetes 58m Normal Started pod/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm Container started 58m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 Container started 58m Normal Created pod/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm Container created 58m Normal Pulled pod/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 Container created 57m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9 Created pod: e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn 57m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-c46397-openshift-default Scaled up replica set e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9 from 0 to 1 57m Normal AddedLabel gateway/e2e-shared-b-w4-c46397 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-c46397 57m Normal AddedInterface pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn Add eth0 [10.133.0.100/23] from ovn-kubernetes 57m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Readiness probe error: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Readiness probe error: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Readiness probe failed: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Readiness probe failed: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-c46397 Scaled up replica set payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d from 0 to 1 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-c46397 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc Created pod: payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d Created pod: payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 57m Warning Unhealthy pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Readiness probe failed: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Readiness probe error: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc Error creating: pods "payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-c46397: serviceaccount "payload-processing-e2e-shared-b-w4-c46397" not found 57m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d Error creating: pods "payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-c46397: serviceaccount "payload-processing-e2e-shared-b-w4-c46397" not found 57m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq Container created 57m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq Add eth0 [10.133.0.101/23] from ovn-kubernetes 57m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq Container started 57m Normal Created pod/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 Container created 57m Normal Pulled pod/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 Add eth0 [10.133.0.102/23] from ovn-kubernetes 57m Normal Started pod/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 Container started 57m Normal Pulled pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Container created 57m Normal Started pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Container started 57m Normal Killing pod/payload-processing-e2e-mt-1f954ca9-5d46f4f796-sh98w Stopping container payload-processing 57m Normal Killing pod/payload-pre-processing-e2e-mt-1f954ca9-549d445bc9-lr5cx Stopping container payload-pre-processing 57m Warning Unhealthy pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 57m Warning ProbeError pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-e11bef-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-e11bef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal Killing pod/e2e-mt-1f954ca9-openshift-default-ffd455d85-hnrnm Stopping container istio-proxy 56m Normal AddedLabel gateway/e2e-mt-c1da2183 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-c1da2183 56m Normal ScalingReplicaSet deployment/e2e-mt-c1da2183-openshift-default Scaled up replica set e2e-mt-c1da2183-openshift-default-746fc598bf from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-mt-c1da2183-openshift-default-746fc598bf Created pod: e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl 56m Normal AddedInterface pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Add eth0 [10.133.0.103/23] from ovn-kubernetes 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-e11bef-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-c1da2183 Scaled up replica set payload-pre-processing-e2e-mt-c1da2183-677d9df459 from 0 to 1 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-e11bef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-c1da2183 Scaled up replica set payload-processing-e2e-mt-c1da2183-8649468f78 from 0 to 1 56m Warning FailedCreate replicaset/payload-processing-e2e-mt-c1da2183-8649468f78 Error creating: pods "payload-processing-e2e-mt-c1da2183-8649468f78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c1da2183: serviceaccount "payload-processing-e2e-mt-c1da2183" not found 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-c1da2183-8649468f78 Created pod: payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-c1da2183-677d9df459 Error creating: pods "payload-pre-processing-e2e-mt-c1da2183-677d9df459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c1da2183: serviceaccount "payload-processing-e2e-mt-c1da2183" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-c1da2183-677d9df459 Created pod: payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl 56m Normal AddedInterface pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl Add eth0 [10.133.0.104/23] from ovn-kubernetes 56m Normal AddedInterface pod/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg Add eth0 [10.133.0.105/23] from ovn-kubernetes 56m Normal Pulled pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl Container created 56m Normal Started pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl Container started 56m Normal Pulled pod/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg Container created 56m Normal Started pod/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg Container started 56m Normal AddedLabel gateway/e2e-shared-a-w3-ad73a4 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-ad73a4 56m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-ad73a4-openshift-default Scaled up replica set e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85 from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85 Created pod: e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv 56m Normal AddedInterface pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Add eth0 [10.133.0.106/23] from ovn-kubernetes 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-c1da2183-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-c1da2183-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55 Created pod: payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-ad73a4: serviceaccount "payload-processing-e2e-shared-a-w3-ad73a4" not found 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8 Created pod: payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-ad73a4 Scaled up replica set payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8 from 0 to 1 56m Normal Pulled pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn Container started 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-ad73a4 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55 from 0 to 1 56m Normal Created pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn Container created 56m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8 Error creating: pods "payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-ad73a4: serviceaccount "payload-processing-e2e-shared-a-w3-ad73a4" not found 56m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f Container started 56m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f Add eth0 [10.134.0.139/23] from ovn-kubernetes 56m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f Container created 56m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq Add eth0 [10.134.0.140/23] from ovn-kubernetes 56m Normal Pulled pod/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq Container created 56m Warning Unhealthy pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 56m Warning ProbeError pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-c46397-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-c46397-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-ad73a4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-ad73a4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-c46397-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-c46397-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning Unhealthy pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Readiness probe failed: Get "http://10.133.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Readiness probe error: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Readiness probe failed: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Readiness probe error: Get "http://10.133.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Normal AddedLabel gateway/e2e-shared-b-w3-70d3a9 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-70d3a9 55m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-70d3a9-openshift-default Scaled up replica set e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5 from 0 to 1 55m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5 Created pod: e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp 55m Normal AddedInterface pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Add eth0 [10.132.0.119/23] from ovn-kubernetes 55m Normal Created pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Container created 55m Normal Pulled pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Container started 55m Normal Created pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Container created 55m Normal Started pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Container started 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-70d3a9 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c from 0 to 1 55m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96 Error creating: pods "payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-70d3a9: serviceaccount "payload-processing-e2e-shared-b-w3-70d3a9" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96 Created pod: payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-70d3a9 Scaled up replica set payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c Created pod: payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-znkvj 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c Error creating: pods "payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-70d3a9: serviceaccount "payload-processing-e2e-shared-b-w3-70d3a9" not found 55m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-znkvj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-znkvj Add eth0 [10.134.0.142/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 Add eth0 [10.134.0.143/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 Container created 55m Normal Started pod/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 Container started 55m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-znkvj Container started 55m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-znkvj Container created 55m Normal SuccessfulCreate replicaset/e2e-api-a-59fb17-openshift-default-84d55cd8fc Created pod: e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx 55m Normal ScalingReplicaSet deployment/e2e-api-a-59fb17-openshift-default Scaled up replica set e2e-api-a-59fb17-openshift-default-84d55cd8fc from 0 to 1 55m Normal AddedLabel gateway/e2e-api-a-59fb17 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-59fb17 55m Normal AddedInterface pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx Add eth0 [10.132.0.120/23] from ovn-kubernetes 55m Normal AddedLabel gateway/e2e-api-a-0e7510 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-0e7510 55m Normal SuccessfulCreate replicaset/e2e-api-a-0e7510-openshift-default-5d8c44ffd9 Created pod: e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 55m Normal AddedInterface pod/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 Add eth0 [10.132.0.121/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/e2e-api-a-0e7510-openshift-default Scaled up replica set e2e-api-a-0e7510-openshift-default-5d8c44ffd9 from 0 to 1 55m Warning ProbeError pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused 55m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-a4782a-openshift-default-948cb9897 Created pod: e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 55m Normal AddedLabel gateway/e2e-shared-a-w2-a4782a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-a4782a 55m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-a4782a-openshift-default Scaled up replica set e2e-shared-a-w2-a4782a-openshift-default-948cb9897 from 0 to 1 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-59fb17 Scaled up replica set payload-processing-e2e-api-a-59fb17-649c79cc74 from 0 to 1 55m Normal Pulled pod/payload-processing-e2e-api-a-59fb17-649c79cc74-jrnbm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-59fb17-89bc69f67-56k2g Add eth0 [10.134.0.145/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-processing-e2e-api-a-59fb17-649c79cc74-jrnbm Add eth0 [10.134.0.144/23] from ovn-kubernetes 55m Warning FailedCreate replicaset/payload-processing-e2e-api-a-59fb17-649c79cc74 Error creating: pods "payload-processing-e2e-api-a-59fb17-649c79cc74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-59fb17: serviceaccount "payload-processing-e2e-api-a-59fb17" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-59fb17-89bc69f67 Created pod: payload-pre-processing-e2e-api-a-59fb17-89bc69f67-56k2g 55m Normal AddedInterface pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Add eth0 [10.132.0.122/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-59fb17 Scaled up replica set payload-pre-processing-e2e-api-a-59fb17-89bc69f67 from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-59fb17-89bc69f67 Error creating: pods "payload-pre-processing-e2e-api-a-59fb17-89bc69f67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-59fb17: serviceaccount "payload-processing-e2e-api-a-59fb17" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-59fb17-649c79cc74 Created pod: payload-processing-e2e-api-a-59fb17-649c79cc74-jrnbm 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-0e7510 Scaled up replica set payload-pre-processing-e2e-api-a-0e7510-646866ffb7 from 0 to 1 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-0e7510 Scaled up replica set payload-processing-e2e-api-a-0e7510-574b9c5d94 from 0 to 1 55m Normal Created pod/payload-pre-processing-e2e-api-a-59fb17-89bc69f67-56k2g Container created 55m Normal Started pod/payload-pre-processing-e2e-api-a-59fb17-89bc69f67-56k2g Container started 55m Normal Started pod/payload-processing-e2e-api-a-59fb17-649c79cc74-jrnbm Container started 55m Normal Pulled pod/payload-pre-processing-e2e-api-a-59fb17-89bc69f67-56k2g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-api-a-59fb17-649c79cc74-jrnbm Container created 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-0e7510-574b9c5d94 Created pod: payload-processing-e2e-api-a-0e7510-574b9c5d94-hvp2q 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-0e7510-646866ffb7 Error creating: pods "payload-pre-processing-e2e-api-a-0e7510-646866ffb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0e7510: serviceaccount "payload-processing-e2e-api-a-0e7510" not found 55m Warning FailedCreate replicaset/payload-processing-e2e-api-a-0e7510-574b9c5d94 Error creating: pods "payload-processing-e2e-api-a-0e7510-574b9c5d94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0e7510: serviceaccount "payload-processing-e2e-api-a-0e7510" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-0e7510-646866ffb7 Created pod: payload-pre-processing-e2e-api-a-0e7510-646866ffb7-t5vrz 55m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-0e7510-646866ffb7-t5vrz Add eth0 [10.134.0.146/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-api-a-0e7510-574b9c5d94-hvp2q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-processing-e2e-api-a-0e7510-574b9c5d94-hvp2q Add eth0 [10.134.0.147/23] from ovn-kubernetes 55m Normal Started pod/payload-processing-e2e-api-a-0e7510-574b9c5d94-hvp2q Container started 55m Normal Started pod/payload-pre-processing-e2e-api-a-0e7510-646866ffb7-t5vrz Container started 55m Normal Created pod/payload-processing-e2e-api-a-0e7510-574b9c5d94-hvp2q Container created 55m Normal Pulled pod/payload-pre-processing-e2e-api-a-0e7510-646866ffb7-t5vrz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-pre-processing-e2e-api-a-0e7510-646866ffb7-t5vrz Container created 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-a4782a-fd478b978 Created pod: payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx 55m Normal Pulled pod/payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-a4782a-fd478b978 Error creating: pods "payload-processing-e2e-shared-a-w2-a4782a-fd478b978-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-a4782a: serviceaccount "payload-processing-e2e-shared-a-w2-a4782a" not found 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-a4782a Scaled up replica set payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c Created pod: payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc 55m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx Add eth0 [10.134.0.148/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-a4782a Scaled up replica set payload-processing-e2e-shared-a-w2-a4782a-fd478b978 from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c Error creating: pods "payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-a4782a: serviceaccount "payload-processing-e2e-shared-a-w2-a4782a" not found 55m Normal Created pod/payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx Container created 55m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc Container started 55m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc Container created 55m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc Add eth0 [10.133.0.110/23] from ovn-kubernetes 55m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx Container started 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-ad73a4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-ad73a4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-c1da2183-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-c1da2183-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-hd6sz Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-hd6sz Add eth0 [10.132.0.123/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-hd6sz 55m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 55m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 55m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-hd6sz Container created 55m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-hd6sz Container started 55m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-hd6sz Readiness probe failed: HTTP probe failed with statuscode: 503 55m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-hd6sz Readiness probe error: HTTP probe failed with statuscode: 503... 54m Normal AddedLabel gateway/e2e-api-b-4266b7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-4266b7 54m Normal ScalingReplicaSet deployment/e2e-api-b-4266b7-openshift-default Scaled up replica set e2e-api-b-4266b7-openshift-default-7f565669bf from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-api-b-4266b7-openshift-default-7f565669bf Created pod: e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 54m Normal SuccessfulCreate replicaset/e2e-api-b-2c7ca6-openshift-default-6554598675 Created pod: e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq 54m Normal AddedInterface pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Add eth0 [10.133.0.117/23] from ovn-kubernetes 54m Normal AddedLabel gateway/e2e-api-b-2c7ca6 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-2c7ca6 54m Normal ScalingReplicaSet deployment/e2e-api-b-2c7ca6-openshift-default Scaled up replica set e2e-api-b-2c7ca6-openshift-default-6554598675 from 0 to 1 54m Normal AddedInterface pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq Add eth0 [10.133.0.118/23] from ovn-kubernetes 54m Normal AddedLabel gateway/e2e-shared-a-w6-908469 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-908469 54m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr 54m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf Created pod: e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 54m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-908469-openshift-default-54f468f759 Created pod: e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 54m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-75c17d-openshift-default Scaled up replica set e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf from 0 to 1 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 7; reason: All metrics below target 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled down replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 10 to 7 54m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-908469-openshift-default Scaled up replica set e2e-shared-a-w6-908469-openshift-default-54f468f759 from 0 to 1 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 8; reason: All metrics below target 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled down replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 10 to 8 54m Normal AddedLabel gateway/e2e-shared-b-w2-75c17d Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-75c17d 54m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 54m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled down replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 10 to 8 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled down replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 10 to 7 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 7; reason: All metrics below target 54m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-2hb4t 54m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-qjfck Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm Stopping container istio-proxy 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 8; reason: All metrics below target 54m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-2wthq 54m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-9nsg9 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj 54m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-7jjpf Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw Stopping container istio-proxy 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 7; reason: All metrics below target 54m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-q5426 54m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ql5qn 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-hq8g9 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-jh5vg 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-d8rvw 54m Normal AddedInterface pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Add eth0 [10.133.0.119/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled down replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 10 to 7 54m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-g9vrm 54m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-n6gtk 54m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled down replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 10 to 7 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 7; reason: All metrics below target 54m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-n89j6 Stopping container istio-proxy 54m Warning ProbeError pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Readiness probe error: Get "http://10.132.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9 Created pod: payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-rflpm 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-4266b7 Scaled up replica set payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9 from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-api-b-4266b7-6f95cbbb78 Error creating: pods "payload-processing-e2e-api-b-4266b7-6f95cbbb78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-4266b7: serviceaccount "payload-processing-e2e-api-b-4266b7" not found 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9 Error creating: pods "payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-4266b7: serviceaccount "payload-processing-e2e-api-b-4266b7" not found 54m Warning ProbeError pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Readiness probe error: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Readiness probe failed: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning Unhealthy pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Readiness probe failed: Get "http://10.132.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal AddedInterface pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Add eth0 [10.133.0.120/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-4266b7 Scaled up replica set payload-processing-e2e-api-b-4266b7-6f95cbbb78 from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-4266b7-6f95cbbb78 Created pod: payload-processing-e2e-api-b-4266b7-6f95cbbb78-s4vfd 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-2c7ca6 Scaled up replica set payload-processing-e2e-api-b-2c7ca6-7cdc8bd888 from 0 to 1 54m Normal Started pod/payload-processing-e2e-api-b-4266b7-6f95cbbb78-s4vfd Container started 54m Normal Created pod/payload-processing-e2e-api-b-4266b7-6f95cbbb78-s4vfd Container created 54m Normal Started pod/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-rflpm Container started 54m Warning FailedCreate replicaset/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888 Error creating: pods "payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-2c7ca6: serviceaccount "payload-processing-e2e-api-b-2c7ca6" not found 54m Normal Created pod/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-rflpm Container created 54m Normal Pulled pod/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-rflpm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Pulled pod/payload-processing-e2e-api-b-4266b7-6f95cbbb78-s4vfd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-2c7ca6 Scaled up replica set payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87 from 0 to 1 54m Normal AddedInterface pod/payload-processing-e2e-api-b-4266b7-6f95cbbb78-s4vfd Add eth0 [10.134.0.151/23] from ovn-kubernetes 54m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-rflpm Add eth0 [10.134.0.152/23] from ovn-kubernetes 54m Normal AddedInterface pod/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-bpjlj Add eth0 [10.134.0.153/23] from ovn-kubernetes 54m Normal Pulled pod/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-bpjlj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-qdztb Add eth0 [10.134.0.154/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-qdztb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888 Created pod: payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-bpjlj 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87 Created pod: payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-qdztb 54m Normal Started pod/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-bpjlj Container started 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87 Error creating: pods "payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-2c7ca6: serviceaccount "payload-processing-e2e-api-b-2c7ca6" not found 54m Normal Created pod/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-bpjlj Container created 54m Normal Created pod/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-qdztb Container created 54m Normal Started pod/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-qdztb Container started 54m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-p6wqr Readiness probe error: HTTP probe failed with statuscode: 503... 54m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Readiness probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-2tbbn Readiness probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused... 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-908469 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f from 0 to 1 54m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Readiness probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-908469-994779fdd Error creating: pods "payload-processing-e2e-shared-a-w6-908469-994779fdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-908469: serviceaccount "payload-processing-e2e-shared-a-w6-908469" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-908469-994779fdd Created pod: payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-908469 Scaled up replica set payload-processing-e2e-shared-a-w6-908469-994779fdd from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f Error creating: pods "payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-908469: serviceaccount "payload-processing-e2e-shared-a-w6-908469" not found 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f Created pod: payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 54m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-hzptj Readiness probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 54m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 Container created 54m Normal Pulled pod/payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 Container started 54m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 Add eth0 [10.134.0.156/23] from ovn-kubernetes 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 Add eth0 [10.134.0.155/23] from ovn-kubernetes 54m Normal Started pod/payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 Container started 54m Normal Created pod/payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 Container created 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-75c17d Scaled up replica set payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95 from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-api-a-59fb17-openshift-default-84d55cd8fc Created pod: e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d: serviceaccount "payload-processing-e2e-shared-b-w2-75c17d" not found 54m Normal SuccessfulCreate replicaset/e2e-api-a-59fb17-openshift-default-84d55cd8fc Created pod: e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-59fb17-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-api-a-59fb17-openshift-default Scaled up replica set e2e-api-a-59fb17-openshift-default-84d55cd8fc from 1 to 4 54m Normal SuccessfulCreate replicaset/e2e-api-a-59fb17-openshift-default-84d55cd8fc Created pod: e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95 Created pod: payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48 Created pod: payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-75c17d Scaled up replica set payload-processing-e2e-shared-b-w2-75c17d-895b5cd48 from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48 Error creating: pods "payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-75c17d: serviceaccount "payload-processing-e2e-shared-b-w2-75c17d" not found 54m Normal Pulled pod/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr Container created 54m Normal Created pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Container created 54m Normal Pulled pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 Container created 54m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr Add eth0 [10.133.0.125/23] from ovn-kubernetes 54m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf Container created 54m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf Add eth0 [10.133.0.124/23] from ovn-kubernetes 54m Normal Started pod/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr Container started 54m Normal AddedInterface pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 Add eth0 [10.133.0.122/23] from ovn-kubernetes 54m Normal Started pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Container started 54m Normal AddedInterface pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 Add eth0 [10.133.0.123/23] from ovn-kubernetes 54m Normal Started pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 Container started 54m Normal Created pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 Container created 54m Normal Pulled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 Container started 54m Normal AddedInterface pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 Add eth0 [10.134.0.157/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 Container started 54m Normal Created pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 Container created 54m Normal Created pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx Container created 54m Normal Pulled pod/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Container created 54m Normal Pulled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 Container started 54m Normal Started pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx Container started 54m Normal Created pod/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 Container created 54m Normal Started pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Container started 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-70d3a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-70d3a9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-a4782a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-a4782a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning ProbeError pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 54m Normal AddedLabel gateway/e2e-api-a-ca18b3 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-ca18b3 54m Warning Unhealthy pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 54m Warning ProbeError pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 54m Normal ScalingReplicaSet deployment/e2e-api-a-ca18b3-openshift-default Scaled up replica set e2e-api-a-ca18b3-openshift-default-68c46d7b5 from 0 to 1 54m Warning Unhealthy pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 Startup probe failed: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused 54m Warning ProbeError pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 Startup probe error: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused... 54m Normal Killing pod/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg Stopping container payload-processing 54m Normal Killing pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl Stopping container payload-pre-processing 54m Normal SuccessfulCreate replicaset/e2e-api-a-ca18b3-openshift-default-68c46d7b5 Created pod: e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c 54m Normal AddedInterface pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Add eth0 [10.133.0.126/23] from ovn-kubernetes 54m Warning Unhealthy pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Startup probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 54m Warning ProbeError pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Startup probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused 54m Warning ProbeError pod/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused... 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59 Created pod: payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-twwlq 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59 Error creating: pods "payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ca18b3: serviceaccount "payload-processing-e2e-api-a-ca18b3" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-ca18b3-668696fdfd Created pod: payload-processing-e2e-api-a-ca18b3-668696fdfd-9w2s9 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-ca18b3 Scaled up replica set payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59 from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-api-a-ca18b3-668696fdfd Error creating: pods "payload-processing-e2e-api-a-ca18b3-668696fdfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ca18b3: serviceaccount "payload-processing-e2e-api-a-ca18b3" not found 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-ca18b3 Scaled up replica set payload-processing-e2e-api-a-ca18b3-668696fdfd from 0 to 1 54m Normal Created pod/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-twwlq Container created 54m Normal Pulled pod/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-twwlq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-twwlq Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-twwlq Add eth0 [10.134.0.158/23] from ovn-kubernetes 54m Normal Started pod/payload-processing-e2e-api-a-ca18b3-668696fdfd-9w2s9 Container started 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-59fb17-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal Created pod/payload-processing-e2e-api-a-ca18b3-668696fdfd-9w2s9 Container created 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-59fb17-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal AddedInterface pod/payload-processing-e2e-api-a-ca18b3-668696fdfd-9w2s9 Add eth0 [10.134.0.159/23] from ovn-kubernetes 54m Normal Pulled pod/payload-processing-e2e-api-a-ca18b3-668696fdfd-9w2s9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-0e7510-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-0e7510-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled down replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 10 to 9 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 9; reason: All metrics below target 54m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-pw8sp Stopping container istio-proxy 53m Warning Unhealthy pod/payload-processing-e2e-mt-c1da2183-8649468f78-7kkmg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 53m Warning Unhealthy pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-7k6cl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-59fb17-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-59fb17-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-70d3a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-70d3a9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled down replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 9 to 6 53m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w 53m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-wv42w Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm 53m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-b9598 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 6; reason: All metrics below target 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-a4782a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-a4782a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-h4twm Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled down replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 7 to 6 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled down replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 7 to 5 53m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled down replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 8 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 6; reason: All metrics below target 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 5; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-pn5fp 53m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled down replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 7 to 5 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-v2d9m 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 6; reason: All metrics below target 53m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-dx6x9 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled down replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 8 to 6 53m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 6; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-jtp64 Stopping container istio-proxy 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 5; reason: All metrics below target 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled down replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 7 to 6 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-6t6s7 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-72ptp 53m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-r2qx9 53m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj 53m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-5th85 53m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-b4gkj Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-7c5v2 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 5; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-5pldb Stopping container istio-proxy 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ca18b3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ca18b3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-59fb17-openshift-default New size: 5; reason: All metrics below target 53m Normal SuccessfulCreate replicaset/e2e-api-a-59fb17-openshift-default-84d55cd8fc Created pod: e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m 53m Normal ScalingReplicaSet deployment/e2e-api-a-59fb17-openshift-default Scaled up replica set e2e-api-a-59fb17-openshift-default-84d55cd8fc from 4 to 5 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-0e7510-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Pulled pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-0e7510-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Created pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m Container created 53m Normal AddedInterface pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m Add eth0 [10.132.0.127/23] from ovn-kubernetes 53m Normal Started pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m Container started 53m Warning Unhealthy pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 53m Warning ProbeError pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 53m Normal AddedLabel gateway/e2e-shared-b-w6-95692f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-95692f 53m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c Created pod: e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 53m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-95692f-openshift-default Scaled up replica set e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c from 0 to 1 53m Normal AddedInterface pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 Add eth0 [10.133.0.129/23] from ovn-kubernetes 53m Normal Killing pod/payload-pre-processing-e2e-api-a-59fb17-89bc69f67-56k2g Stopping container payload-pre-processing 53m Normal Killing pod/payload-processing-e2e-api-a-59fb17-649c79cc74-jrnbm Stopping container payload-processing 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-95692f Scaled up replica set payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959 Created pod: payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-95692f: serviceaccount "payload-processing-e2e-shared-b-w6-95692f" not found 53m Normal AddedLabel gateway/e2e-inf-a-c31999 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c31999 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b Created pod: payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w 53m Normal ScalingReplicaSet deployment/e2e-inf-a-c31999-openshift-default Scaled up replica set e2e-inf-a-c31999-openshift-default-967c87455 from 0 to 1 53m Normal SuccessfulCreate replicaset/e2e-inf-a-c31999-openshift-default-967c87455 Created pod: e2e-inf-a-c31999-openshift-default-967c87455-k4v94 53m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b Error creating: pods "payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-95692f: serviceaccount "payload-processing-e2e-shared-b-w6-95692f" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-95692f Scaled up replica set payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959 from 0 to 1 53m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w Add eth0 [10.134.0.164/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Pulled pod/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p Container created 53m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p Add eth0 [10.134.0.163/23] from ovn-kubernetes 53m Normal Started pod/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w Container started 53m Normal Created pod/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w Container created 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-c1da2183-677d9df459 Created pod: payload-pre-processing-e2e-mt-c1da2183-677d9df459-hm4ww 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-c1da2183-8649468f78 Created pod: payload-processing-e2e-mt-c1da2183-8649468f78-r2ncj 53m Warning FailedCreate replicaset/payload-processing-e2e-mt-c1da2183-8649468f78 Error creating: pods "payload-processing-e2e-mt-c1da2183-8649468f78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c1da2183: serviceaccount "payload-processing-e2e-mt-c1da2183" not found 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-c1da2183-677d9df459 Error creating: pods "payload-pre-processing-e2e-mt-c1da2183-677d9df459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c1da2183: serviceaccount "payload-processing-e2e-mt-c1da2183" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-c1da2183 Scaled up replica set payload-pre-processing-e2e-mt-c1da2183-677d9df459 from 0 to 1 53m Normal AddedInterface pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Add eth0 [10.133.0.130/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-c1da2183 Scaled up replica set payload-processing-e2e-mt-c1da2183-8649468f78 from 0 to 1 53m Normal Created pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-hm4ww Container created 53m Normal AddedInterface pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-hm4ww Add eth0 [10.134.0.165/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-processing-e2e-mt-c1da2183-8649468f78-r2ncj Add eth0 [10.134.0.166/23] from ovn-kubernetes 53m Normal Killing pod/payload-processing-e2e-api-a-0e7510-574b9c5d94-hvp2q Stopping container payload-processing 53m Normal Pulled pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-hm4ww Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-hm4ww Container started 53m Normal Killing pod/payload-pre-processing-e2e-api-a-0e7510-646866ffb7-t5vrz Stopping container payload-pre-processing 53m Normal Pulled pod/payload-processing-e2e-mt-c1da2183-8649468f78-r2ncj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-processing-e2e-mt-c1da2183-8649468f78-r2ncj Container started 53m Normal Created pod/payload-processing-e2e-mt-c1da2183-8649468f78-r2ncj Container created 53m Normal SuccessfulCreate replicaset/e2e-api-a-ca18b3-openshift-default-68c46d7b5 Created pod: e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-ca18b3-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-api-a-ca18b3-openshift-default-68c46d7b5 Created pod: e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 53m Normal SuccessfulCreate replicaset/e2e-api-a-ca18b3-openshift-default-68c46d7b5 Created pod: e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z 53m Normal ScalingReplicaSet deployment/e2e-api-a-ca18b3-openshift-default Scaled up replica set e2e-api-a-ca18b3-openshift-default-68c46d7b5 from 1 to 4 53m Normal Pulled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 Container created 53m Normal Created pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z Container created 53m Normal Started pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 Container started 53m Normal Pulled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867 Error creating: pods "payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c31999: serviceaccount "payload-processing-e2e-inf-a-c31999" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867 Created pod: payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 53m Normal AddedInterface pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk Add eth0 [10.133.0.131/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c31999 Scaled up replica set payload-pre-processing-e2e-inf-a-c31999-57f7b8f867 from 0 to 1 53m Normal AddedInterface pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z Add eth0 [10.132.0.130/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c31999 Scaled up replica set payload-processing-e2e-inf-a-c31999-74bcfbbd68 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c31999-74bcfbbd68 Created pod: payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp 53m Normal Pulled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z Container started 53m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c31999-74bcfbbd68 Error creating: pods "payload-processing-e2e-inf-a-c31999-74bcfbbd68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c31999: serviceaccount "payload-processing-e2e-inf-a-c31999" not found 53m Normal AddedInterface pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 Add eth0 [10.134.0.167/23] from ovn-kubernetes 53m Normal Started pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk Container started 53m Normal Started pod/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 Container started 53m Normal Created pod/payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp Container created 53m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 Add eth0 [10.134.0.169/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp Add eth0 [10.134.0.168/23] from ovn-kubernetes 53m Normal Created pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk Container created 53m Normal Started pod/payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp Container started 53m Normal Created pod/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 Container created 53m Warning ProbeError pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Readiness probe error: Get "http://10.133.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Readiness probe failed: Get "http://10.133.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Readiness probe error: Get "http://10.133.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Readiness probe failed: Get "http://10.133.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Readiness probe error: Get "http://10.133.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Readiness probe failed: Get "http://10.133.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Normal Pulled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Container created 52m Normal Pulled pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Container created 52m Normal Started pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Container started 52m Normal Started pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Container started 52m Normal Created pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Container created 52m Normal Pulled pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Container started 52m Normal Started pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq Container started 52m Normal Created pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq Container created 52m Normal Pulled pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedLabel gateway/e2e-api-b-517366 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-517366 52m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/e2e-api-b-517366-openshift-default-8555fc7c7f Created pod: e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s 52m Normal ScalingReplicaSet deployment/e2e-api-b-517366-openshift-default Scaled up replica set e2e-api-b-517366-openshift-default-8555fc7c7f from 0 to 1 52m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused... 52m Normal ScalingReplicaSet deployment/e2e-mt-c1da2183-openshift-default Scaled up replica set e2e-mt-c1da2183-openshift-default-746fc598bf from 1 to 2 52m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused 52m Normal SuccessfulCreate replicaset/e2e-mt-c1da2183-openshift-default-746fc598bf Created pod: e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-c1da2183-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal Created pod/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv Container created 52m Normal AddedInterface pod/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv Add eth0 [10.132.0.131/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv Container started 52m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk Startup probe error: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk Startup probe failed: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused 52m Normal AddedInterface pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Add eth0 [10.133.0.132/23] from ovn-kubernetes 52m Warning ProbeError pod/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv Startup probe error: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv Startup probe failed: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-75c17d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-75c17d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ca18b3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ca18b3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning ProbeError pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 52m Warning ProbeError pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Startup probe error: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Startup probe failed: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Startup probe error: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Startup probe failed: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused 52m Warning ProbeError pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused... 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-517366-86d679464c Error creating: pods "payload-pre-processing-e2e-api-b-517366-86d679464c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-517366: serviceaccount "payload-processing-e2e-api-b-517366" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-517366-7c5b497f8d Created pod: payload-processing-e2e-api-b-517366-7c5b497f8d-nrz2x 52m Warning FailedCreate replicaset/payload-processing-e2e-api-b-517366-7c5b497f8d Error creating: pods "payload-processing-e2e-api-b-517366-7c5b497f8d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-517366: serviceaccount "payload-processing-e2e-api-b-517366" not found 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-517366 Scaled up replica set payload-pre-processing-e2e-api-b-517366-86d679464c from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-517366 Scaled up replica set payload-processing-e2e-api-b-517366-7c5b497f8d from 0 to 1 52m Normal Created pod/payload-pre-processing-e2e-api-b-517366-86d679464c-rh6sb Container created 52m Normal Pulled pod/payload-processing-e2e-api-b-517366-7c5b497f8d-nrz2x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-api-b-517366-7c5b497f8d-nrz2x Container created 52m Normal Started pod/payload-processing-e2e-api-b-517366-7c5b497f8d-nrz2x Container started 52m Normal AddedInterface pod/payload-processing-e2e-api-b-517366-7c5b497f8d-nrz2x Add eth0 [10.134.0.170/23] from ovn-kubernetes 52m Normal Pulled pod/payload-pre-processing-e2e-api-b-517366-86d679464c-rh6sb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Started pod/payload-pre-processing-e2e-api-b-517366-86d679464c-rh6sb Container started 52m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-517366-86d679464c-rh6sb Add eth0 [10.134.0.171/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-517366-86d679464c Created pod: payload-pre-processing-e2e-api-b-517366-86d679464c-rh6sb 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-2c7ca6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-2c7ca6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-908469-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-908469-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal SuccessfulCreate replicaset/e2e-api-a-ca18b3-openshift-default-68c46d7b5 Created pod: e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-ca18b3-openshift-default New size: 7; reason: All metrics below target 52m Normal ScalingReplicaSet deployment/e2e-api-a-ca18b3-openshift-default Scaled up replica set e2e-api-a-ca18b3-openshift-default-68c46d7b5 from 4 to 7 52m Normal SuccessfulCreate replicaset/e2e-api-a-ca18b3-openshift-default-68c46d7b5 Created pod: e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk 52m Normal SuccessfulCreate replicaset/e2e-api-a-ca18b3-openshift-default-68c46d7b5 Created pod: e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 52m Normal Created pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk Container created 52m Normal AddedInterface pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk Add eth0 [10.132.0.133/23] from ovn-kubernetes 52m Normal Started pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 Container started 52m Normal Pulled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 Container created 52m Normal AddedInterface pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 Add eth0 [10.133.0.133/23] from ovn-kubernetes 52m Normal Started pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 Container started 52m Normal Started pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk Container started 52m Normal AddedInterface pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 Add eth0 [10.134.0.174/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 Container created 52m Normal Pulled pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 Startup probe error: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 Startup probe error: Get "http://10.134.0.174:15021/healthz/ready": dial tcp 10.134.0.174:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 Startup probe failed: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 Startup probe failed: Get "http://10.134.0.174:15021/healthz/ready": dial tcp 10.134.0.174:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-517366-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-517366-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-2c7ca6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-2c7ca6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-908469-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-908469-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-75c17d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-75c17d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal Killing pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-gm5q7 Stopping container istio-proxy 52m Normal Killing pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-c96xx Stopping container istio-proxy 52m Normal Killing pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-xwn5m Stopping container istio-proxy 52m Normal Killing pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-kjm57 Stopping container istio-proxy 52m Normal Killing pod/e2e-api-a-59fb17-openshift-default-84d55cd8fc-6rr54 Stopping container istio-proxy 52m Normal Killing pod/e2e-api-a-0e7510-openshift-default-5d8c44ffd9-zl775 Stopping container istio-proxy 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Readiness probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Readiness probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Readiness probe error: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Readiness probe failed: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Normal Killing pod/payload-pre-processing-e2e-api-b-4266b7-5b54d5d5f9-rflpm Stopping container payload-pre-processing 52m Normal Killing pod/payload-processing-e2e-api-b-4266b7-6f95cbbb78-s4vfd Stopping container payload-processing 52m Normal Killing pod/payload-processing-e2e-api-b-2c7ca6-7cdc8bd888-bpjlj Stopping container payload-processing 52m Normal Killing pod/payload-pre-processing-e2e-api-b-2c7ca6-79759dcb87-qdztb Stopping container payload-pre-processing 51m Normal Pulled pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 Container started 51m Normal Created pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 Container created 51m Warning ProbeError pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Readiness probe error: Get "http://10.133.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Readiness probe failed: Get "http://10.133.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused 51m Warning ProbeError pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused... 51m Normal Created pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Container created 51m Normal Started pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Container started 51m Normal Pulled pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedLabel gateway/e2e-inf-a-1eab65 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-1eab65 51m Normal SuccessfulCreate replicaset/e2e-inf-a-1eab65-openshift-default-7c57f54487 Created pod: e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj 51m Normal AddedInterface pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj Add eth0 [10.132.0.137/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-inf-a-1eab65-openshift-default Scaled up replica set e2e-inf-a-1eab65-openshift-default-7c57f54487 from 0 to 1 51m Warning Unhealthy pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Startup probe failed: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused 51m Warning ProbeError pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Startup probe error: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused... 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-4266b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-4266b7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887 Created pod: payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-1eab65 Scaled up replica set payload-processing-e2e-inf-a-1eab65-8576d668f6 from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-1eab65 Scaled up replica set payload-pre-processing-e2e-inf-a-1eab65-6c8485c887 from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887 Error creating: pods "payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-1eab65: serviceaccount "payload-processing-e2e-inf-a-1eab65" not found 51m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-1eab65-8576d668f6 Error creating: pods "payload-processing-e2e-inf-a-1eab65-8576d668f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-1eab65: serviceaccount "payload-processing-e2e-inf-a-1eab65" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-1eab65-8576d668f6 Created pod: payload-processing-e2e-inf-a-1eab65-8576d668f6-2lxn5 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-95692f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Started pod/payload-processing-e2e-inf-a-1eab65-8576d668f6-2lxn5 Container started 51m Normal Created pod/payload-processing-e2e-inf-a-1eab65-8576d668f6-2lxn5 Container created 51m Normal Pulled pod/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 Add eth0 [10.133.0.135/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-inf-a-1eab65-8576d668f6-2lxn5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-95692f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal AddedInterface pod/payload-processing-e2e-inf-a-1eab65-8576d668f6-2lxn5 Add eth0 [10.134.0.177/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 Container started 51m Normal ScalingReplicaSet deployment/e2e-inf-b-093daa-openshift-default Scaled up replica set e2e-inf-b-093daa-openshift-default-7cd449db7c from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-inf-b-093daa-openshift-default-7cd449db7c Created pod: e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt 51m Normal AddedLabel gateway/e2e-inf-b-093daa Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-093daa 51m Normal Created pod/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 Container created 51m Normal Killing pod/payload-pre-processing-e2e-mt-c1da2183-677d9df459-hm4ww Stopping container payload-pre-processing 51m Normal Killing pod/payload-processing-e2e-mt-c1da2183-8649468f78-r2ncj Stopping container payload-processing 51m Normal AddedInterface pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt Add eth0 [10.133.0.136/23] from ovn-kubernetes 51m Warning ProbeError pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Readiness probe error: Get "http://10.133.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal ScalingReplicaSet deployment/e2e-inf-a-c31999-openshift-default Scaled up replica set e2e-inf-a-c31999-openshift-default-967c87455 from 1 to 3 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-c31999-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-inf-a-c31999-openshift-default-967c87455 Created pod: e2e-inf-a-c31999-openshift-default-967c87455-lxzhs 51m Normal SuccessfulCreate replicaset/e2e-inf-a-c31999-openshift-default-967c87455 Created pod: e2e-inf-a-c31999-openshift-default-967c87455-pshs8 51m Warning Unhealthy pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Readiness probe failed: Get "http://10.133.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal Created pod/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs Container created 51m Normal AddedInterface pod/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 Add eth0 [10.132.0.138/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs Container started 51m Normal AddedInterface pod/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs Add eth0 [10.134.0.178/23] from ovn-kubernetes 51m Normal Started pod/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 Container started 51m Normal Created pod/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 Container created 51m Normal Pulled pod/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-093daa Scaled up replica set payload-processing-e2e-inf-b-093daa-5d4fd9f674 from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-093daa-5d4fd9f674 Error creating: pods "payload-processing-e2e-inf-b-093daa-5d4fd9f674-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-093daa: serviceaccount "payload-processing-e2e-inf-b-093daa" not found 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-093daa-878f6f547 Created pod: payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-093daa-878f6f547 Error creating: pods "payload-pre-processing-e2e-inf-b-093daa-878f6f547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-093daa: serviceaccount "payload-processing-e2e-inf-b-093daa" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-093daa-5d4fd9f674 Created pod: payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-093daa Scaled up replica set payload-pre-processing-e2e-inf-b-093daa-878f6f547 from 0 to 1 51m Normal Started pod/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 Container started 51m Normal Created pod/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 Container created 51m Normal Started pod/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 Container started 51m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 Add eth0 [10.133.0.138/23] from ovn-kubernetes 51m Normal Created pod/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 Container created 51m Normal Pulled pod/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 Add eth0 [10.133.0.137/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Container started 51m Normal Created pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Container created 51m Normal Started pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Container started 51m Normal Created pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Container created 51m Normal Pulled pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs Startup probe failed: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused 51m Warning ProbeError pod/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs Startup probe error: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 Startup probe error: Get "http://10.132.0.138:15021/healthz/ready": dial tcp 10.132.0.138:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 Startup probe failed: Get "http://10.132.0.138:15021/healthz/ready": dial tcp 10.132.0.138:15021: connect: connection refused 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-95692f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-95692f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning ProbeError pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Startup probe error: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Startup probe failed: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused 51m Warning ProbeError pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Startup probe error: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Startup probe failed: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c31999-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c31999-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-1eab65-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-1eab65-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal ScalingReplicaSet deployment/e2e-inf-a-79c24a-openshift-default Scaled up replica set e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f from 0 to 1 51m Normal AddedLabel gateway/e2e-inf-a-79c24a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-79c24a 51m Normal SuccessfulCreate replicaset/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f Created pod: e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l 51m Normal AddedInterface pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l Add eth0 [10.133.0.139/23] from ovn-kubernetes 51m Normal Killing pod/payload-processing-e2e-api-a-ca18b3-668696fdfd-9w2s9 Stopping container payload-processing 51m Normal Killing pod/payload-pre-processing-e2e-api-a-ca18b3-5c8bc94c59-twwlq Stopping container payload-pre-processing 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-4266b7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-4266b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-79c24a Scaled up replica set payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-79c24a Scaled up replica set payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8 from 0 to 1 51m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 Add eth0 [10.134.0.180/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8 Created pod: payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc Created pod: payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 51m Normal Pulled pod/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 Container started 51m Normal Created pod/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 Container created 51m Normal Pulled pod/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg Container created 51m Normal Started pod/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg Container started 51m Normal AddedInterface pod/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg Add eth0 [10.134.0.179/23] from ovn-kubernetes 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc Error creating: pods "payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-79c24a: serviceaccount "payload-processing-e2e-inf-a-79c24a" not found 51m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8 Error creating: pods "payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-79c24a: serviceaccount "payload-processing-e2e-inf-a-79c24a" not found 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c31999-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c31999-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Killing pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Stopping container istio-proxy 50m Normal SuccessfulCreate replicaset/e2e-inf-a-79e911-openshift-default-67558899b7 Created pod: e2e-inf-a-79e911-openshift-default-67558899b7-2b97d 50m Normal ScalingReplicaSet deployment/e2e-inf-a-79e911-openshift-default Scaled up replica set e2e-inf-a-79e911-openshift-default-67558899b7 from 0 to 1 50m Normal AddedLabel gateway/e2e-inf-a-79e911 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-79e911 50m Warning Unhealthy pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Readiness probe failed: HTTP probe failed with statuscode: 503 50m Normal Killing pod/e2e-api-b-2c7ca6-openshift-default-6554598675-rnmkq Stopping container istio-proxy 50m Warning ProbeError pod/e2e-api-b-4266b7-openshift-default-7f565669bf-5jzh2 Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal AddedInterface pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Add eth0 [10.133.0.140/23] from ovn-kubernetes 50m Normal AddedLabel gateway/e2e-inf-a-c5145c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c5145c 50m Normal ScalingReplicaSet deployment/e2e-inf-a-c5145c-openshift-default Scaled up replica set e2e-inf-a-c5145c-openshift-default-6cb9588fc7 from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-inf-a-c5145c-openshift-default-6cb9588fc7 Created pod: e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-517366-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-517366-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal AddedInterface pod/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r Add eth0 [10.132.0.140/23] from ovn-kubernetes 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-79e911-6dc57c86dd Error creating: pods "payload-processing-e2e-inf-a-79e911-6dc57c86dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-79e911: serviceaccount "payload-processing-e2e-inf-a-79e911" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-79e911-6dc57c86dd Created pod: payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-79e911-65d94b4767 Error creating: pods "payload-pre-processing-e2e-inf-a-79e911-65d94b4767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-79e911: serviceaccount "payload-processing-e2e-inf-a-79e911" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-79e911-65d94b4767 Created pod: payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-79e911 Scaled up replica set payload-pre-processing-e2e-inf-a-79e911-65d94b4767 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-79e911 Scaled up replica set payload-processing-e2e-inf-a-79e911-6dc57c86dd from 0 to 1 50m Normal Created pod/payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr Container created 50m Normal Pulled pod/payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc Container started 50m Normal Started pod/payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr Container started 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr Add eth0 [10.134.0.184/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc Container created 50m Normal AddedInterface pod/payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc Add eth0 [10.134.0.183/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c5145c-5488c5dd7d Created pod: payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c5145c Scaled up replica set payload-processing-e2e-inf-a-c5145c-5488c5dd7d from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f Error creating: pods "payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c5145c: serviceaccount "payload-processing-e2e-inf-a-c5145c" not found 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c5145c Scaled up replica set payload-pre-processing-e2e-inf-a-c5145c-6c5746747f from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f Created pod: payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c5145c-5488c5dd7d Error creating: pods "payload-processing-e2e-inf-a-c5145c-5488c5dd7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c5145c: serviceaccount "payload-processing-e2e-inf-a-c5145c" not found 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd Add eth0 [10.133.0.142/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5 Add eth0 [10.133.0.141/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5 Container started 50m Normal Created pod/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd Container created 50m Normal Started pod/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd Container started 50m Normal Created pod/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5 Container created 50m Normal Killing pod/e2e-mt-c1da2183-openshift-default-746fc598bf-gghdv Stopping container istio-proxy 50m Normal Killing pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-inf-b-290ede-openshift-default Scaled up replica set e2e-inf-b-290ede-openshift-default-cf8d7764c from 0 to 1 50m Normal AddedLabel gateway/e2e-inf-b-290ede Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-290ede 50m Normal SuccessfulCreate replicaset/e2e-inf-b-290ede-openshift-default-cf8d7764c Created pod: e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm 50m Warning ProbeError pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal AddedInterface pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Add eth0 [10.133.0.143/23] from ovn-kubernetes 50m Warning Unhealthy pod/e2e-mt-c1da2183-openshift-default-746fc598bf-nlpcl Readiness probe failed: HTTP probe failed with statuscode: 503 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled down replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 5 to 4 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 4; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-fwxv2 Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-290ede Scaled up replica set payload-processing-e2e-inf-b-290ede-564cc79644 from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59 Error creating: pods "payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-290ede: serviceaccount "payload-processing-e2e-inf-b-290ede" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59 Created pod: payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-290ede-564cc79644 Error creating: pods "payload-processing-e2e-inf-b-290ede-564cc79644-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-290ede: serviceaccount "payload-processing-e2e-inf-b-290ede" not found 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-290ede Scaled up replica set payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-290ede-564cc79644 Created pod: payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg 50m Normal AddedInterface pod/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg Add eth0 [10.133.0.145/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx Container started 50m Normal Created pod/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg Container created 50m Normal Pulled pod/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx Container created 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx Add eth0 [10.133.0.144/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg Container started 50m Normal AddedLabel gateway/e2e-inf-a-b2628a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-b2628a 50m Normal ScalingReplicaSet deployment/e2e-inf-a-b2628a-openshift-default Scaled up replica set e2e-inf-a-b2628a-openshift-default-5898ccb7f7 from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-inf-a-b2628a-openshift-default-5898ccb7f7 Created pod: e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 50m Normal Killing pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-qrgjk Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-4f27z Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-k5dq4 Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-lvlsk Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-8bt22 Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-7ftj2 Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 50m Normal AddedInterface pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Add eth0 [10.133.0.146/23] from ovn-kubernetes 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 5; reason: All metrics below target 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default Scaled down replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 6 to 5 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 5; reason: All metrics below target 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj 50m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf 50m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-pzv92 Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled down replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 6 to 5 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 5; reason: All metrics below target 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled down replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 6 to 5 50m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-j6lqf Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Stopping container istio-proxy 50m Warning Unhealthy pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-api-a-ca18b3-openshift-default-68c46d7b5-xwk4c Readiness probe error: HTTP probe failed with statuscode: 503... 49m Normal SuccessfulCreate replicaset/e2e-inf-a-1eab65-openshift-default-7c57f54487 Created pod: e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl 49m Normal ScalingReplicaSet deployment/e2e-inf-a-1eab65-openshift-default Scaled up replica set e2e-inf-a-1eab65-openshift-default-7c57f54487 from 1 to 4 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-1eab65-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-inf-a-1eab65-openshift-default-7c57f54487 Created pod: e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx 49m Normal SuccessfulCreate replicaset/e2e-inf-a-1eab65-openshift-default-7c57f54487 Created pod: e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d 49m Normal Started pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d Container started 49m Normal Created pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx Container created 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f Error creating: pods "payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b2628a: serviceaccount "payload-processing-e2e-inf-a-b2628a" not found 49m Normal AddedInterface pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d Add eth0 [10.134.0.186/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj Container created 49m Normal Pulled pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx Container started 49m Normal Pulled pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d Container created 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-b2628a Scaled up replica set payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f from 0 to 1 49m Normal SuccessfulCreate replicaset/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f Created pod: e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx 49m Normal Started pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj Container started 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f Created pod: payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-b2628a Scaled up replica set payload-processing-e2e-inf-a-b2628a-657b857bd6 from 0 to 1 49m Normal ScalingReplicaSet deployment/e2e-inf-a-79c24a-openshift-default Scaled up replica set e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f from 1 to 4 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-79c24a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f Created pod: e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-b2628a-657b857bd6 Created pod: payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 49m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-b2628a-657b857bd6 Error creating: pods "payload-processing-e2e-inf-a-b2628a-657b857bd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b2628a: serviceaccount "payload-processing-e2e-inf-a-b2628a" not found 49m Normal Started pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl Container started 49m Normal AddedInterface pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl Add eth0 [10.133.0.148/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f Created pod: e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc 49m Normal AddedInterface pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx Add eth0 [10.132.0.143/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl Container created 49m Normal Created pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq Container created 49m Normal AddedInterface pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx Add eth0 [10.133.0.149/23] from ovn-kubernetes 49m Normal Started pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx Container started 49m Normal AddedInterface pod/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 Add eth0 [10.133.0.151/23] from ovn-kubernetes 49m Normal Created pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx Container created 49m Normal Pulled pod/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h Container started 49m Normal Pulled pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 Container created 49m Normal Started pod/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 Container started 49m Normal Created pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc Container created 49m Normal Created pod/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h Container created 49m Normal AddedInterface pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq Add eth0 [10.134.0.187/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc Add eth0 [10.132.0.144/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq Container started 49m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h Add eth0 [10.133.0.150/23] from ovn-kubernetes 49m Normal Started pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc Container started 49m Normal Pulled pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Readiness probe failed: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-8tphj Readiness probe error: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal Pulled pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l Container created 49m Normal Started pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l Container started 49m Normal Pulled pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt Container started 49m Normal Created pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt Container created 49m Normal Killing pod/payload-pre-processing-e2e-api-b-517366-86d679464c-rh6sb Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-api-b-517366-7c5b497f8d-nrz2x Stopping container payload-processing 49m Warning ProbeError pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d Startup probe error: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d Startup probe failed: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq Startup probe failed: Get "http://10.134.0.187:15021/healthz/ready": dial tcp 10.134.0.187:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq Startup probe error: Get "http://10.134.0.187:15021/healthz/ready": dial tcp 10.134.0.187:15021: connect: connection refused... 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-093daa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-093daa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning Unhealthy pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc Startup probe failed: Get "http://10.132.0.144:15021/healthz/ready": dial tcp 10.132.0.144:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc Startup probe error: Get "http://10.132.0.144:15021/healthz/ready": dial tcp 10.132.0.144:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj Startup probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj Startup probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx Startup probe error: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl Startup probe failed: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx Startup probe failed: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl Startup probe error: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-79c24a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-79c24a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal AddedLabel gateway/e2e-inf-b-3b28bc Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-3b28bc 49m Normal SuccessfulCreate replicaset/e2e-inf-b-3b28bc-openshift-default-7b668b8685 Created pod: e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl 49m Normal ScalingReplicaSet deployment/e2e-inf-b-3b28bc-openshift-default Scaled up replica set e2e-inf-b-3b28bc-openshift-default-7b668b8685 from 0 to 1 49m Normal AddedInterface pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Add eth0 [10.133.0.152/23] from ovn-kubernetes 49m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd Error creating: pods "payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-3b28bc: serviceaccount "payload-processing-e2e-inf-b-3b28bc" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-3b28bc Scaled up replica set payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9 from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9 Error creating: pods "payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-3b28bc: serviceaccount "payload-processing-e2e-inf-b-3b28bc" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd Created pod: payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-2t48r 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-3b28bc Scaled up replica set payload-processing-e2e-inf-b-3b28bc-6bb979c6fd from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9 Created pod: payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg 49m Normal Pulled pod/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-2t48r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg Add eth0 [10.134.0.188/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg Container created 49m Normal Started pod/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-2t48r Container started 49m Normal AddedInterface pod/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-2t48r Add eth0 [10.134.0.189/23] from ovn-kubernetes 49m Normal Created pod/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-2t48r Container created 49m Normal Started pod/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg Container started 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-1eab65-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-1eab65-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-79c24a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-79c24a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-093daa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-093daa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-hd6sz 49m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 49m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-hd6sz Stopping container discovery 49m Normal AddedLabel gateway/e2e-inf-b-c9477c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-c9477c 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 4; reason: All metrics below target 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled down replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 6 to 4 49m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w Stopping container istio-proxy 49m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 49m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw Stopping container istio-proxy 49m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-rln8w 49m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-qc4rw 49m Normal SuccessfulCreate replicaset/e2e-inf-b-c9477c-openshift-default-548875cd79 Created pod: e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x 49m Normal ScalingReplicaSet deployment/e2e-inf-b-c9477c-openshift-default Scaled up replica set e2e-inf-b-c9477c-openshift-default-548875cd79 from 0 to 1 49m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Stopping container istio-proxy 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled down replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 5 to 4 49m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z Stopping container istio-proxy 49m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g 49m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default Scaled down replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 5 to 3 49m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-ktx2z 49m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default Scaled down replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 5 to 4 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 4; reason: All metrics below target 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 3; reason: All metrics below target 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default New size: 4; reason: All metrics below target 49m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n Stopping container istio-proxy 49m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-phj9g Stopping container istio-proxy 49m Normal AddedInterface pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Add eth0 [10.132.0.147/23] from ovn-kubernetes 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 4; reason: All metrics below target 49m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz Stopping container istio-proxy 49m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-89czz 49m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default Scaled down replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 5 to 4 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 4; reason: All metrics below target 49m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-fmch2 Stopping container istio-proxy 49m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-gcq8n 48m Normal AddedLabel gateway/e2e-inf-b-200d11 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-200d11 48m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Readiness probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-bx4g5 Readiness probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 48m Normal SuccessfulCreate replicaset/e2e-inf-b-200d11-openshift-default-5cb455cdfc Created pod: e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss 48m Normal ScalingReplicaSet deployment/e2e-inf-b-200d11-openshift-default Scaled up replica set e2e-inf-b-200d11-openshift-default-5cb455cdfc from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9 Error creating: pods "payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c9477c: serviceaccount "payload-processing-e2e-inf-b-c9477c" not found 48m Normal AddedInterface pod/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss Add eth0 [10.132.0.148/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9 Created pod: payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-c9477c Scaled up replica set payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-c9477c Scaled up replica set payload-processing-e2e-inf-b-c9477c-bff6d4884 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-c9477c-bff6d4884 Created pod: payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-c9477c-bff6d4884 Error creating: pods "payload-processing-e2e-inf-b-c9477c-bff6d4884-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c9477c: serviceaccount "payload-processing-e2e-inf-b-c9477c" not found 48m Normal Pulled pod/payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 Add eth0 [10.132.0.149/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 Container started 48m Normal Created pod/payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk Container created 48m Normal Started pod/payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk Container started 48m Normal Created pod/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 Container created 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk Add eth0 [10.132.0.150/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-200d11-5f9d5865f8 Created pod: payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-200d11-5f9d5865f8 Error creating: pods "payload-processing-e2e-inf-b-200d11-5f9d5865f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-200d11: serviceaccount "payload-processing-e2e-inf-b-200d11" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-200d11 Scaled up replica set payload-pre-processing-e2e-inf-b-200d11-99fbc69b7 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7 Created pod: payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7 Error creating: pods "payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-200d11: serviceaccount "payload-processing-e2e-inf-b-200d11" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-200d11 Scaled up replica set payload-processing-e2e-inf-b-200d11-5f9d5865f8 from 0 to 1 48m Warning ProbeError pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Readiness probe error: Get "http://10.133.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Started pod/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk Container started 48m Normal Started pod/payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj Container started 48m Normal Created pod/payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj Container created 48m Normal Pulled pod/payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Readiness probe failed: Get "http://10.133.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk Add eth0 [10.132.0.151/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj Add eth0 [10.132.0.152/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk Container created 48m Normal Created pod/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r Container created 48m Normal Started pod/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r Container started 48m Normal Pulled pod/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Container started 48m Normal Created pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Container created 48m Normal Created pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Container created 48m Normal Started pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Container started 48m Normal Pulled pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-79e911-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-79e911-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c5145c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c5145c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Startup probe error: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Startup probe failed: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-290ede-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-290ede-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-093daa-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-inf-b-093daa-openshift-default Scaled up replica set e2e-inf-b-093daa-openshift-default-7cd449db7c from 1 to 2 48m Normal SuccessfulCreate replicaset/e2e-inf-b-093daa-openshift-default-7cd449db7c Created pod: e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 48m Normal AddedInterface pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Add eth0 [10.132.0.154/23] from ovn-kubernetes 48m Normal Started pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Container started 48m Normal Pulled pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Container created 48m Warning Unhealthy pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Startup probe failed: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused 48m Warning ProbeError pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Startup probe error: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Readiness probe failed: Get "http://10.133.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Readiness probe error: Get "http://10.133.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Created pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Container created 48m Normal Pulled pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Container started 48m Normal Started pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Container started 48m Normal Pulled pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Container created 48m Normal AddedLabel gateway/e2e-inf-b-abf04b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-abf04b 48m Normal ScalingReplicaSet deployment/e2e-inf-b-abf04b-openshift-default Scaled up replica set e2e-inf-b-abf04b-openshift-default-74ccb66d7f from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-b-abf04b-openshift-default-74ccb66d7f Created pod: e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c5145c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c5145c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal AddedInterface pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Add eth0 [10.132.0.156/23] from ovn-kubernetes 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-290ede-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-290ede-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal Killing pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Stopping container istio-proxy 48m Normal AddedLabel gateway/e2e-inf-a-67f8e0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-67f8e0 48m Normal ScalingReplicaSet deployment/e2e-inf-a-67f8e0-openshift-default Scaled up replica set e2e-inf-a-67f8e0-openshift-default-954489f5f from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-a-67f8e0-openshift-default-954489f5f Created pod: e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b2628a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b2628a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Startup probe error: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 48m Normal AddedInterface pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Add eth0 [10.133.0.154/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Startup probe failed: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-abf04b-c4db67488 Error creating: pods "payload-processing-e2e-inf-b-abf04b-c4db67488-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-abf04b: serviceaccount "payload-processing-e2e-inf-b-abf04b" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-abf04b-c4db67488 Created pod: payload-processing-e2e-inf-b-abf04b-c4db67488-6jzt4 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-abf04b Scaled up replica set payload-processing-e2e-inf-b-abf04b-c4db67488 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-abf04b-566d865847 Error creating: pods "payload-pre-processing-e2e-inf-b-abf04b-566d865847-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-abf04b: serviceaccount "payload-processing-e2e-inf-b-abf04b" not found 48m Warning ProbeError pod/e2e-api-b-517366-openshift-default-8555fc7c7f-h6q5s Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-abf04b Scaled up replica set payload-pre-processing-e2e-inf-b-abf04b-566d865847 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-abf04b-566d865847 Created pod: payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb 48m Normal Pulled pod/payload-processing-e2e-inf-b-abf04b-c4db67488-6jzt4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb Container started 48m Normal Started pod/payload-processing-e2e-inf-b-abf04b-c4db67488-6jzt4 Container started 48m Normal Created pod/payload-processing-e2e-inf-b-abf04b-c4db67488-6jzt4 Container created 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-abf04b-c4db67488-6jzt4 Add eth0 [10.134.0.192/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb Container created 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb Add eth0 [10.134.0.193/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-67f8e0 Scaled up replica set payload-processing-e2e-inf-a-67f8e0-6d5854845b from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-67f8e0-6d5854845b Created pod: payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-67f8e0 Scaled up replica set payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67 from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-67f8e0-6d5854845b Error creating: pods "payload-processing-e2e-inf-a-67f8e0-6d5854845b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-67f8e0: serviceaccount "payload-processing-e2e-inf-a-67f8e0" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67 Created pod: payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67 Error creating: pods "payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-67f8e0: serviceaccount "payload-processing-e2e-inf-a-67f8e0" not found 47m Normal Created pod/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg Container created 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q Add eth0 [10.134.0.194/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg Container started 47m Normal Started pod/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q Container started 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg Add eth0 [10.133.0.155/23] from ovn-kubernetes 47m Normal Created pod/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q Container created 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-3b28bc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-3b28bc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-79e911-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-79e911-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg 47m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default Scaled down replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 4 to 3 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 3; reason: All metrics below target 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 3; reason: All metrics below target 47m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-lm4vn 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled down replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 4 to 3 47m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Readiness probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-pcgrg Readiness probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-3b28bc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-3b28bc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b2628a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b2628a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-c9477c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 46m Normal AddedInterface pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Add eth0 [10.133.0.166/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Add eth0 [10.134.0.204/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-inf-b-c9477c-openshift-default-548875cd79 Created pod: e2e-inf-b-c9477c-openshift-default-548875cd79-45slr 46m Normal SuccessfulCreate replicaset/e2e-inf-b-c9477c-openshift-default-548875cd79 Created pod: e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp 46m Normal SuccessfulCreate replicaset/e2e-inf-b-c9477c-openshift-default-548875cd79 Created pod: e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk 46m Normal ScalingReplicaSet deployment/e2e-inf-b-c9477c-openshift-default Scaled up replica set e2e-inf-b-c9477c-openshift-default-548875cd79 from 1 to 4 46m Normal Pulled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Container created 46m Normal Pulled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Container started 46m Normal Created pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Container created 46m Normal Started pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Container started 46m Normal Created pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Container created 46m Normal Pulled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Container started 46m Normal AddedInterface pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Add eth0 [10.133.0.165/23] from ovn-kubernetes 46m Normal Started pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Container started 46m Normal Created pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Container created 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-vp9n6 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-57f78bcd55 from 10 to 9 46m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-vp9n6 Stopping container istio-proxy 46m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Startup probe failed: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Startup probe error: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Startup probe failed: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Startup probe error: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Startup probe error: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Startup probe failed: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Startup probe failed: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Startup probe error: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused... 46m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-thdbs 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 3; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default Scaled down replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 4 to 3 46m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default New size: 3; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-xgv78 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql 46m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-xr6ql Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default New size: 3; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-57f78bcd55 from 9 to 6 46m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-m9l2r Stopping container istio-proxy 46m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-2tsvw Stopping container istio-proxy 46m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-7kdrm Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-2tsvw 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-m9l2r 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-7kdrm 46m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-zhrnf Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Readiness probe error: Get "http://10.132.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Readiness probe failed: Get "http://10.132.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal Pulled pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Container created 46m Normal Started pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Container started 46m Normal Pulled pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c9477c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning Unhealthy pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Readiness probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c9477c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning ProbeError pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Readiness probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 46m Normal Started pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Container started 46m Normal Created pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Container created 46m Warning Unhealthy pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Startup probe failed: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Startup probe error: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused... 46m Normal ScalingReplicaSet deployment/e2e-inf-b-3b28bc-openshift-default Scaled up replica set e2e-inf-b-3b28bc-openshift-default-7b668b8685 from 1 to 2 46m Warning Unhealthy pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 46m Normal SuccessfulCreate replicaset/e2e-inf-b-3b28bc-openshift-default-7b668b8685 Created pod: e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-3b28bc-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 46m Warning ProbeError pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 46m Normal AddedInterface pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Add eth0 [10.132.0.177/23] from ovn-kubernetes 46m Normal Started pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Container started 46m Normal Created pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Container created 46m Normal Pulled pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal SuccessfulCreate replicaset/e2e-inf-b-f23eae-openshift-default-6bcbf599c Created pod: e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr 46m Normal AddedLabel gateway/e2e-inf-b-f23eae Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-f23eae 46m Normal ScalingReplicaSet deployment/e2e-inf-b-f23eae-openshift-default Scaled up replica set e2e-inf-b-f23eae-openshift-default-6bcbf599c from 0 to 1 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-67f8e0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-67f8e0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal AddedInterface pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr Add eth0 [10.133.0.167/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Add eth0 [10.132.0.178/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-inf-b-c9477c-openshift-default-548875cd79 Created pod: e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh 46m Normal Pulled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal SuccessfulCreate replicaset/e2e-inf-b-c9477c-openshift-default-548875cd79 Created pod: e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 46m Normal ScalingReplicaSet deployment/e2e-inf-b-c9477c-openshift-default Scaled up replica set e2e-inf-b-c9477c-openshift-default-548875cd79 from 4 to 7 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-c9477c-openshift-default New size: 7; reason: All metrics below target 46m Warning ProbeError pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Startup probe error: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Startup probe failed: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused 46m Normal AddedInterface pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Add eth0 [10.134.0.206/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Add eth0 [10.133.0.169/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-inf-b-c9477c-openshift-default-548875cd79 Created pod: e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn 46m Normal Created pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Container created 46m Normal Created pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Container created 46m Normal Started pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Container started 46m Normal Started pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Container started 46m Normal Created pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Container created 46m Normal Pulled pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Container started 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-f23eae-86bbf64bdb Error creating: pods "payload-processing-e2e-inf-b-f23eae-86bbf64bdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f23eae: serviceaccount "payload-processing-e2e-inf-b-f23eae" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7 Error creating: pods "payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f23eae: serviceaccount "payload-processing-e2e-inf-b-f23eae" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-f23eae Scaled up replica set payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-f23eae-86bbf64bdb Created pod: payload-processing-e2e-inf-b-f23eae-86bbf64bdb-cxtqq 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7 Created pod: payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-f23eae Scaled up replica set payload-processing-e2e-inf-b-f23eae-86bbf64bdb from 0 to 1 46m Normal Pulled pod/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-inf-b-f23eae-86bbf64bdb-cxtqq Container started 46m Normal Pulled pod/payload-processing-e2e-inf-b-f23eae-86bbf64bdb-cxtqq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-inf-b-f23eae-86bbf64bdb-cxtqq Add eth0 [10.134.0.207/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm Container started 46m Normal Created pod/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm Container created 46m Normal Created pod/payload-processing-e2e-inf-b-f23eae-86bbf64bdb-cxtqq Container created 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm Add eth0 [10.134.0.208/23] from ovn-kubernetes 45m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Startup probe failed: Get "http://10.134.0.206:15021/healthz/ready": dial tcp 10.134.0.206:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Startup probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused 45m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Startup probe error: Get "http://10.134.0.206:15021/healthz/ready": dial tcp 10.134.0.206:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Startup probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Startup probe failed: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused 45m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Startup probe error: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused... 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-abf04b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-abf04b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-67f8e0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-67f8e0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 2; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Stopping container istio-proxy 45m Normal Created pod/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss Container created 45m Normal Pulled pod/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 3 to 2 45m Normal Started pod/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss Container started 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm 45m Warning ProbeError pod/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-200d11-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Readiness probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-200d11-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-ncgmm Readiness probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-200d11-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-200d11-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Started pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr Container started 44m Normal Pulled pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr Container created 44m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc 44m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-zfzdq 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default New size: 2; reason: All metrics below target 44m Warning Unhealthy pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr Startup probe failed: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused 44m Warning ProbeError pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr Startup probe error: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-dmdbg Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-tf4lc Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f23eae-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f23eae-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default New size: 2; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-vt687 44m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Stopping container istio-proxy 44m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-jmz2v Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f23eae-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f23eae-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default New size: 2; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-n67hw Stopping container istio-proxy 43m Normal Killing pod/payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 Stopping container payload-pre-processing 43m Warning Unhealthy pod/payload-processing-e2e-inf-a-c31999-74bcfbbd68-j77lp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 43m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c31999-57f7b8f867-8z889 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-lkltg 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-2ff29 42m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 42m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-2ff29 Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-57f78bcd55 from 6 to 4 42m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-lkltg Stopping container istio-proxy 42m Normal Killing pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Stopping container istio-proxy 42m Normal Killing pod/e2e-inf-a-c31999-openshift-default-967c87455-pshs8 Stopping container istio-proxy 42m Normal Killing pod/e2e-inf-a-c31999-openshift-default-967c87455-lxzhs Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-dtvjx 42m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 42m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-dtvjx Stopping container discovery 42m Warning Unhealthy pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-inf-a-c31999-openshift-default-967c87455-k4v94 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal Killing pod/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 Stopping container payload-pre-processing 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-1eab65-openshift-default New size: 3; reason: All metrics below target 42m Normal Killing pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-inf-a-1eab65-openshift-default Scaled down replica set e2e-inf-a-1eab65-openshift-default-7c57f54487 from 4 to 3 42m Normal SuccessfulDelete replicaset/e2e-inf-a-1eab65-openshift-default-7c57f54487 Deleted pod: e2e-inf-a-1eab65-openshift-default-7c57f54487-4kdcj 42m Warning Unhealthy pod/payload-processing-e2e-inf-b-093daa-5d4fd9f674-8prq7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-093daa-878f6f547-lrg78 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Normal Killing pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-zrkbt Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-inf-b-093daa-openshift-default-7cd449db7c-8njc5 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal ScalingReplicaSet deployment/e2e-inf-a-67f8e0-openshift-default Scaled up replica set e2e-inf-a-67f8e0-openshift-default-954489f5f from 1 to 2 41m Normal SuccessfulCreate replicaset/e2e-inf-a-67f8e0-openshift-default-954489f5f Created pod: e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-67f8e0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal Created pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Container created 41m Normal AddedInterface pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Add eth0 [10.132.0.186/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Container started 41m Warning ProbeError pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Startup probe error: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Startup probe failed: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused 41m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-cdm7z 41m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-57f78bcd55 from 4 to 3 41m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-cdm7z Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal Killing pod/payload-processing-e2e-inf-a-1eab65-8576d668f6-2lxn5 Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 Stopping container payload-pre-processing 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-43c070-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-43c070-openshift-default-857c6bc8df from 2 to 1 40m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-43c070-openshift-default-857c6bc8df Deleted pod: e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-43c070-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-1eab65-6c8485c887-2xgx6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-hdwzh Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal Killing pod/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-inf-a-c5145c-5488c5dd7d-g58n5 Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd Stopping container payload-pre-processing 40m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-a4782a-openshift-default-948cb9897 Created pod: e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw 40m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-a4782a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-58fea9-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-a4782a-openshift-default Scaled up replica set e2e-shared-a-w2-a4782a-openshift-default-948cb9897 from 1 to 2 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-58fea9-openshift-default-7b68988877 Deleted pod: e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-58fea9-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-58fea9-openshift-default-7b68988877 from 2 to 1 40m Normal Created pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Container created 40m Normal Pulled pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Container started 40m Normal AddedInterface pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Add eth0 [10.133.0.173/23] from ovn-kubernetes 40m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-56ac74-openshift-default-859dc8d675 Deleted pod: e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-56ac74-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-56ac74-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-56ac74-openshift-default-859dc8d675 from 2 to 1 40m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-wzvb6 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal Killing pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-4zm2d Stopping container istio-proxy 40m Normal Killing pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-8htpl Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-qwc64 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal Killing pod/e2e-inf-a-1eab65-openshift-default-7c57f54487-qq6vx Stopping container istio-proxy 40m Warning ProbeError pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Startup probe error: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Startup probe failed: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused 40m Normal ScalingReplicaSet deployment/e2e-inf-b-abf04b-openshift-default Scaled up replica set e2e-inf-b-abf04b-openshift-default-74ccb66d7f from 1 to 2 40m Normal SuccessfulCreate replicaset/e2e-inf-b-abf04b-openshift-default-74ccb66d7f Created pod: e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-abf04b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal Started pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh Container started 40m Normal Created pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh Container created 40m Normal Pulled pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh Add eth0 [10.133.0.174/23] from ovn-kubernetes 40m Warning Unhealthy pod/payload-processing-e2e-inf-a-79c24a-5bcf5c7bd8-gsvxg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-79c24a-666f6f8fbc-5fw55 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal Killing pod/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg Stopping container payload-processing 40m Warning ProbeError pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh Startup probe error: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh Startup probe failed: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused 40m Normal Killing pod/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx Stopping container payload-pre-processing 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-839163-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2cda44-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c5145c-6c5746747f-xhqqd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w4-839163-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-839163-openshift-default-ff4d9b8f from 2 to 1 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-839163-openshift-default-ff4d9b8f Deleted pod: e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt 40m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-2cda44-openshift-default-9c988549c Deleted pod: e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-2cda44-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-2cda44-openshift-default-9c988549c from 2 to 1 39m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-mvsnt Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning ProbeError pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-rhx8c Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/e2e-inf-a-c5145c-openshift-default-6cb9588fc7-j4m8r Stopping container istio-proxy 39m Warning Unhealthy pod/payload-processing-e2e-inf-b-290ede-564cc79644-vbwrg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-290ede-7f5c48cd59-k2xbx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-sbtlx Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-6z57l Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-jsspq Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-a-79c24a-openshift-default-6bdc7cfc7f-vwphc Stopping container istio-proxy 39m Normal Killing pod/payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk Stopping container payload-pre-processing 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-79477e-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-79477e-openshift-default-55bb567445 from 2 to 1 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-79477e-openshift-default-55bb567445 Deleted pod: e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-79477e-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Stopping container istio-proxy 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-inf-b-3b28bc-6bb979c6fd-2t48r Stopping container payload-processing 39m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-x9r4v Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Killing pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Stopping container istio-proxy 39m Warning Unhealthy pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-inf-b-290ede-openshift-default-cf8d7764c-p7rjm Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/payload-processing-e2e-inf-b-200d11-5f9d5865f8-qx6rj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-200d11-99fbc69b7-b54sk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc Stopping container payload-processing 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-3b28bc-5fbdf66cf9-8q4qg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Stopping container istio-proxy 39m Warning ProbeError pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-inf-a-79e911-openshift-default-67558899b7-2b97d Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h Stopping container payload-pre-processing 38m Normal Killing pod/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk Stopping container payload-processing 38m Normal Killing pod/e2e-inf-b-200d11-openshift-default-5cb455cdfc-6rqss Stopping container istio-proxy 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-79e911-65d94b4767-glrjr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5 Created pod: e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j 38m Warning Unhealthy pod/payload-processing-e2e-inf-a-79e911-6dc57c86dd-p2bjc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal AddedLabel gateway/e2e-shared-a-w1-f5bef0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-f5bef0 38m Normal AddedInterface pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j Add eth0 [10.132.0.205/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-f5bef0-openshift-default Scaled up replica set e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5 from 0 to 1 38m Normal Killing pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Stopping container istio-proxy 38m Warning ProbeError pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-t6rvl Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-f5bef0 Scaled up replica set payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-f5bef0: serviceaccount "payload-processing-e2e-shared-a-w1-f5bef0" not found 38m Normal Pulled pod/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx Container created 38m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx Container started 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf Created pod: e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-75c17d-openshift-default Scaled up replica set e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf from 1 to 2 38m Normal Created pod/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv Container created 38m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv Add eth0 [10.132.0.206/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-f5bef0 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7 from 0 to 1 38m Normal Started pod/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv Container started 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-75c17d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Warning Unhealthy pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78 Error creating: pods "payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-f5bef0: serviceaccount "payload-processing-e2e-shared-a-w1-f5bef0" not found 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx Add eth0 [10.134.0.212/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78 Created pod: payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv 38m Warning ProbeError pod/e2e-inf-b-3b28bc-openshift-default-7b668b8685-4lf7v Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7 Created pod: payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx 38m Normal Pulled pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 Container created 38m Normal AddedInterface pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 Add eth0 [10.132.0.207/23] from ovn-kubernetes 38m Normal Started pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 Container started 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-b2628a-8c94cf65f-j6x6h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/payload-processing-e2e-inf-a-b2628a-657b857bd6-4bgh7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 Startup probe failed: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 Startup probe error: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused... 38m Normal Killing pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Stopping container istio-proxy 38m Warning ProbeError pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-inf-a-b2628a-openshift-default-5898ccb7f7-2h6h7 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning Unhealthy pod/payload-processing-e2e-inf-b-c9477c-bff6d4884-l56hk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-c9477c-6f4b5bc9-shcq7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal Killing pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-66cc54-openshift-default-745899c56f Created pod: e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm 38m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal AddedLabel gateway/e2e-shared-a-w0-66cc54 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-66cc54 38m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-gknmh Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-bqstn Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-l2k56 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Killing pod/payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-inf-b-abf04b-c4db67488-6jzt4 Stopping container payload-processing 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-66cc54-openshift-default Scaled up replica set e2e-shared-a-w0-66cc54-openshift-default-745899c56f from 0 to 1 38m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-6zbxp Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal AddedInterface pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Add eth0 [10.132.0.210/23] from ovn-kubernetes 38m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-45slr Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-zkh4x Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning ProbeError pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-inf-b-c9477c-openshift-default-548875cd79-vd6fk Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879 Created pod: payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-66cc54: serviceaccount "payload-processing-e2e-shared-a-w0-66cc54" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-66cc54 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-66cc54 Scaled up replica set payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d Created pod: payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d Error creating: pods "payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-66cc54: serviceaccount "payload-processing-e2e-shared-a-w0-66cc54" not found 38m Normal Created pod/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb Container created 38m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv Container started 38m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv Container created 38m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv Add eth0 [10.133.0.175/23] from ovn-kubernetes 38m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb Add eth0 [10.134.0.214/23] from ovn-kubernetes 38m Normal Pulled pod/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb Container started 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-abf04b-566d865847-m6kgb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal Killing pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-dmpdh Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Stopping container istio-proxy 37m Warning Unhealthy pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-inf-b-abf04b-openshift-default-74ccb66d7f-glxm9 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg Stopping container payload-processing 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-889195-openshift-default-5b79476475 Created pod: e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-889195-openshift-default Scaled up replica set e2e-shared-b-w1-889195-openshift-default-5b79476475 from 0 to 1 37m Normal AddedLabel gateway/e2e-shared-b-w1-889195 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-889195 37m Normal AddedInterface pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh Add eth0 [10.133.0.178/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-889195 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-889195: serviceaccount "payload-processing-e2e-shared-b-w1-889195" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-889195-66b9c594bd Error creating: pods "payload-processing-e2e-shared-b-w1-889195-66b9c594bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-889195: serviceaccount "payload-processing-e2e-shared-b-w1-889195" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-889195-66b9c594bd Created pod: payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-889195 Scaled up replica set payload-processing-e2e-shared-b-w1-889195-66b9c594bd from 0 to 1 37m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn Container created 37m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn Container started 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767 Created pod: payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn 37m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-67f8e0-55b7f4dc67-2jd6q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn Add eth0 [10.134.0.216/23] from ovn-kubernetes 37m Warning Unhealthy pod/payload-processing-e2e-inf-a-67f8e0-6d5854845b-qhffg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal Started pod/payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 Container started 37m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 Add eth0 [10.132.0.213/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 Container created 37m Normal Created pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j Container created 37m Normal Pulled pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j Container started 37m Warning Unhealthy pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Stopping container istio-proxy 37m Warning ProbeError pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-kzctb Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Stopping container istio-proxy 37m Warning ProbeError pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-inf-a-67f8e0-openshift-default-954489f5f-8bwd7 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j Startup probe error: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused... 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-f5bef0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-f5bef0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning Unhealthy pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j Startup probe failed: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused 37m Normal AddedLabel gateway/e2e-shared-b-w0-9cd83f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-9cd83f 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-9cd83f-openshift-default Scaled up replica set e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc Created pod: e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 37m Normal Killing pod/payload-processing-e2e-inf-b-f23eae-86bbf64bdb-cxtqq Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm Stopping container payload-pre-processing 37m Normal AddedInterface pod/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 Add eth0 [10.132.0.215/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-9cd83f Scaled up replica set payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d Created pod: payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq Add eth0 [10.134.0.218/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h Container created 37m Normal Started pod/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h Container started 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d Error creating: pods "payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f: serviceaccount "payload-processing-e2e-shared-b-w0-9cd83f" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-9cd83f Scaled up replica set payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c Created pod: payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq 37m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq Container created 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c Error creating: pods "payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-9cd83f: serviceaccount "payload-processing-e2e-shared-b-w0-9cd83f" not found 37m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h Add eth0 [10.133.0.180/23] from ovn-kubernetes 37m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq Container started 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-95692f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-95692f-openshift-default Scaled up replica set e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c from 1 to 2 37m Normal AddedInterface pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm Add eth0 [10.132.0.216/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm Container created 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c Created pod: e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm 37m Normal Started pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm Container started 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-f23eae-68dd4785d7-7rccm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm Startup probe failed: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm Startup probe error: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused... 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-f5bef0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-f5bef0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/e2e-inf-b-f23eae-openshift-default-6bcbf599c-zncxr Stopping container istio-proxy 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c Created pod: e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-55c5e0-openshift-default Scaled up replica set e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c from 0 to 1 36m Normal AddedLabel gateway/e2e-shared-a-w5-55c5e0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-55c5e0 36m Normal AddedInterface pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Add eth0 [10.133.0.181/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-55c5e0 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9 from 0 to 1 36m Normal Pulled pod/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-55c5e0 Scaled up replica set payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb Created pod: payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb Error creating: pods "payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-55c5e0: serviceaccount "payload-processing-e2e-shared-a-w5-55c5e0" not found 36m Normal Started pod/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq Container started 36m Normal Created pod/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq Container created 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-55c5e0: serviceaccount "payload-processing-e2e-shared-a-w5-55c5e0" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9 Created pod: payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc 36m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq Add eth0 [10.134.0.220/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc Add eth0 [10.133.0.182/23] from ovn-kubernetes 36m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc Container started 36m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc Container created 36m Normal Pulled pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Container created 36m Normal Started pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Container started 36m Warning Unhealthy pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Startup probe failed: Get "http://10.132.0.210:15021/healthz/ready": dial tcp 10.132.0.210:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Startup probe error: Get "http://10.132.0.210:15021/healthz/ready": dial tcp 10.132.0.210:15021: connect: connection refused... 36m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-57f78bcd55 from 3 to 2 36m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-4nxfb 36m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-66cc54-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-66cc54-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-4nxfb Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-55c5e0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-55c5e0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-66cc54-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-66cc54-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Started pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh Container started 35m Normal Pulled pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh Container created 35m Normal Created pod/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 Container created 35m Normal Started pod/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 Container started 35m Normal Pulled pod/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh Startup probe error: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh Startup probe failed: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 Startup probe failed: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 Startup probe error: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused... 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-889195-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-889195-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7 Created pod: e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-da52fd-openshift-default Scaled up replica set e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7 from 0 to 1 35m Normal AddedLabel gateway/e2e-shared-b-w5-da52fd Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-da52fd 35m Normal AddedInterface pod/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf Add eth0 [10.132.0.217/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-da52fd Scaled up replica set payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-da52fd Scaled up replica set payload-processing-e2e-shared-b-w5-da52fd-8567bc8548 from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548 Error creating: pods "payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-da52fd: serviceaccount "payload-processing-e2e-shared-b-w5-da52fd" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548 Created pod: payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-da52fd: serviceaccount "payload-processing-e2e-shared-b-w5-da52fd" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79 Created pod: payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h 35m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h Container started 35m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q Add eth0 [10.134.0.224/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h Container created 35m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h Add eth0 [10.134.0.223/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q Container started 35m Normal Created pod/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q Container created 35m Normal Pulled pod/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-9cd83f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-9cd83f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-889195-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-889195-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Pulled pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Container started 35m Normal Created pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Container created 35m Warning ProbeError pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-55c5e0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-55c5e0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Started pod/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf Container started 34m Normal Pulled pod/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf Container created 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-da52fd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-da52fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning Unhealthy pod/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf Startup probe failed: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf Startup probe error: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused... 34m Normal SuccessfulCreate replicaset/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7 Created pod: e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx 34m Normal ScalingReplicaSet deployment/e2e-ipp-a-cdcf9d-openshift-default Scaled up replica set e2e-ipp-a-cdcf9d-openshift-default-97867d6c7 from 0 to 1 34m Normal AddedLabel gateway/e2e-ipp-a-cdcf9d Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-cdcf9d 34m Normal AddedInterface pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx Add eth0 [10.133.0.188/23] from ovn-kubernetes 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c Error creating: pods "payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-cdcf9d: serviceaccount "payload-processing-e2e-ipp-a-cdcf9d" not found 34m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d Error creating: pods "payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-cdcf9d: serviceaccount "payload-processing-e2e-ipp-a-cdcf9d" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d Created pod: payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-mdttb 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-cdcf9d Scaled up replica set payload-processing-e2e-ipp-a-cdcf9d-cc578d77d from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-cdcf9d Scaled up replica set payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c Created pod: payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-p2pvx 34m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-p2pvx Add eth0 [10.134.0.227/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-p2pvx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-mdttb Container started 34m Normal AddedInterface pod/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-mdttb Add eth0 [10.132.0.221/23] from ovn-kubernetes 34m Normal Created pod/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-p2pvx Container created 34m Normal Started pod/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-p2pvx Container started 34m Normal Pulled pod/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-mdttb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-mdttb Container created 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-da52fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-da52fd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c Created pod: e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-55c5e0-openshift-default Scaled up replica set e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c from 1 to 2 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-55c5e0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal Created pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Container created 33m Normal Started pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Container started 33m Normal AddedInterface pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Add eth0 [10.132.0.223/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning ProbeError pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Startup probe error: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Startup probe failed: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused 32m Normal Killing pod/payload-processing-e2e-ipp-a-cdcf9d-cc578d77d-mdttb Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-ipp-a-cdcf9d-5cb9bc454c-p2pvx Stopping container payload-pre-processing 32m Normal Started pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx Container started 32m Normal Created pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx Container created 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-cdcf9d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Pulled pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-cdcf9d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning ProbeError pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx Startup probe error: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx Startup probe failed: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d Created pod: payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-cdcf9d Scaled up replica set payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c Error creating: pods "payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-cdcf9d: serviceaccount "payload-processing-e2e-ipp-a-cdcf9d" not found 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d Error creating: pods "payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-cdcf9d: serviceaccount "payload-processing-e2e-ipp-a-cdcf9d" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c Created pod: payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-cdcf9d Scaled up replica set payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d from 0 to 1 32m Normal Pulling pod/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 32m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Stopping container discovery 32m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 Add eth0 [10.134.0.230/23] from ovn-kubernetes 32m Normal AddedInterface pod/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 Add eth0 [10.134.0.231/23] from ovn-kubernetes 32m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-8s4mz 32m Normal Started pod/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 Container started 32m Normal Created pod/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 Container created 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.09s (2.09s including waiting). Image size: 166612641 bytes. 32m Normal Pulled pod/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.893s (1.893s including waiting). Image size: 166612641 bytes. 32m Normal Started pod/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 Container started 32m Normal Created pod/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 Container created 32m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-worker-w6-ed088d-openshift-default Scaled down replica set e2e-worker-w6-ed088d-openshift-default-6d4f744f5 from 2 to 1 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ed088d-openshift-default New size: 1; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-worker-w6-ed088d-openshift-default-6d4f744f5 Deleted pod: e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v 32m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Readiness probe failed: Get "http://10.132.0.114:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-zbl8v Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-8s4mz Readiness probe error: Get "http://10.132.0.114:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 31m Normal SuccessfulCreate replicaset/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c Created pod: e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs 31m Normal ScalingReplicaSet deployment/e2e-ipp-b-c3bbba-openshift-default Scaled up replica set e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c from 0 to 1 31m Normal AddedLabel gateway/e2e-ipp-b-c3bbba Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-c3bbba 31m Normal AddedInterface pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Add eth0 [10.132.0.227/23] from ovn-kubernetes 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-cdcf9d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-cdcf9d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-c3bbba Scaled up replica set payload-processing-e2e-ipp-b-c3bbba-55dddc5984 from 0 to 1 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-c3bbba Scaled up replica set payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff from 0 to 1 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff Error creating: pods "payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c3bbba: serviceaccount "payload-processing-e2e-ipp-b-c3bbba" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-c3bbba-55dddc5984 Created pod: payload-processing-e2e-ipp-b-c3bbba-55dddc5984-9dxtt 31m Normal Started pod/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-9zjb7 Container started 31m Normal Created pod/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-9zjb7 Container created 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-9zjb7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-9zjb7 Add eth0 [10.132.0.229/23] from ovn-kubernetes 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-c3bbba-55dddc5984 Error creating: pods "payload-processing-e2e-ipp-b-c3bbba-55dddc5984-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c3bbba: serviceaccount "payload-processing-e2e-ipp-b-c3bbba" not found 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff Created pod: payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-9zjb7 31m Normal Started pod/payload-processing-e2e-ipp-b-c3bbba-55dddc5984-9dxtt Container started 31m Normal Created pod/payload-processing-e2e-ipp-b-c3bbba-55dddc5984-9dxtt Container created 31m Normal Pulled pod/payload-processing-e2e-ipp-b-c3bbba-55dddc5984-9dxtt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-processing-e2e-ipp-b-c3bbba-55dddc5984-9dxtt Add eth0 [10.132.0.230/23] from ovn-kubernetes 31m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 1; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-57f78bcd55 from 2 to 1 31m Normal Killing pod/maas-default-gateway-openshift-default-57f78bcd55-p8tnk Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-57f78bcd55 Deleted pod: maas-default-gateway-openshift-default-57f78bcd55-p8tnk 31m Normal SuccessfulCreate replicaset/e2e-ipp-a-203209-openshift-default-6f6fdc6799 Created pod: e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 31m Normal AddedLabel gateway/e2e-ipp-a-203209 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-203209 31m Normal ScalingReplicaSet deployment/e2e-ipp-a-203209-openshift-default Scaled up replica set e2e-ipp-a-203209-openshift-default-6f6fdc6799 from 0 to 1 31m Normal AddedInterface pod/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 Add eth0 [10.132.0.247/23] from ovn-kubernetes 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967 Error creating: pods "payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-203209: serviceaccount "payload-processing-e2e-ipp-a-203209" not found 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-203209 Scaled up replica set payload-pre-processing-e2e-ipp-a-203209-64dfbfc967 from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967 Created pod: payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-203209-d84fc668d Error creating: pods "payload-processing-e2e-ipp-a-203209-d84fc668d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-203209: serviceaccount "payload-processing-e2e-ipp-a-203209" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-203209-d84fc668d Created pod: payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-203209 Scaled up replica set payload-processing-e2e-ipp-a-203209-d84fc668d from 0 to 1 31m Normal Started pod/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f Container started 31m Normal Pulled pod/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f Container created 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn Add eth0 [10.133.0.193/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f Add eth0 [10.133.0.192/23] from ovn-kubernetes 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn Container created 31m Normal Started pod/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn Container started 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-908469-openshift-default-54f468f759 Created pod: e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-908469-openshift-default Scaled up replica set e2e-shared-a-w6-908469-openshift-default-54f468f759 from 1 to 2 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-908469-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal Pulled pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg Container started 30m Normal Created pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg Container created 30m Normal AddedInterface pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg Add eth0 [10.132.0.249/23] from ovn-kubernetes 30m Warning Unhealthy pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg Startup probe failed: Get "http://10.132.0.249:15021/healthz/ready": dial tcp 10.132.0.249:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg Startup probe error: Get "http://10.132.0.249:15021/healthz/ready": dial tcp 10.132.0.249:15021: connect: connection refused... 30m Normal Killing pod/payload-processing-e2e-ipp-b-c3bbba-55dddc5984-9dxtt Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-ipp-b-c3bbba-6586575dff-9zjb7 Stopping container payload-pre-processing 29m Normal Created pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Container created 29m Normal Pulled pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 Container started 29m Normal Started pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Container started 29m Normal Pulled pod/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 Container created 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-c3bbba Scaled up replica set payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c from 0 to 1 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-203209-64dfbfc967-zm4qn Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-ipp-a-203209-d84fc668d-nw84f Stopping container payload-processing 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-c3bbba Scaled up replica set payload-processing-e2e-ipp-b-c3bbba-76456ccc86 from 0 to 1 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-c3bbba-76456ccc86 Error creating: pods "payload-processing-e2e-ipp-b-c3bbba-76456ccc86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c3bbba: serviceaccount "payload-processing-e2e-ipp-b-c3bbba" not found 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c Error creating: pods "payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c3bbba: serviceaccount "payload-processing-e2e-ipp-b-c3bbba" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c Created pod: payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-c3bbba-76456ccc86 Created pod: payload-processing-e2e-ipp-b-c3bbba-76456ccc86-d86kz 29m Normal Pulling pod/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 29m Normal AddedInterface pod/payload-processing-e2e-ipp-b-c3bbba-76456ccc86-d86kz Add eth0 [10.134.0.239/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl Add eth0 [10.133.0.194/23] from ovn-kubernetes 29m Normal Created pod/payload-processing-e2e-ipp-b-c3bbba-76456ccc86-d86kz Container created 29m Normal Pulled pod/payload-processing-e2e-ipp-b-c3bbba-76456ccc86-d86kz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-processing-e2e-ipp-b-c3bbba-76456ccc86-d86kz Container started 29m Normal Started pod/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl Container started 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.56s (1.56s including waiting). Image size: 166612641 bytes. 29m Normal Created pod/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl Container created 29m Warning Unhealthy pod/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 Startup probe failed: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused 29m Warning ProbeError pod/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 Startup probe error: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused... 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-203209-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning ProbeError pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Startup probe error: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Startup probe failed: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-203209-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal AddedInterface pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Add eth0 [10.133.0.195/23] from ovn-kubernetes 29m Normal AddedLabel gateway/e2e-ipp-a-e1d376 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-e1d376 29m Normal ScalingReplicaSet deployment/e2e-ipp-a-e1d376-openshift-default Scaled up replica set e2e-ipp-a-e1d376-openshift-default-77d6b898c from 0 to 1 29m Normal SuccessfulCreate replicaset/e2e-ipp-a-e1d376-openshift-default-77d6b898c Created pod: e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e1d376 Scaled up replica set payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e1d376 Scaled up replica set payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8 from 0 to 1 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls Container created 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls Container started 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79 Error creating: pods "payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e1d376: serviceaccount "payload-processing-e2e-ipp-a-e1d376" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79 Created pod: payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8 Created pod: payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8 Error creating: pods "payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e1d376: serviceaccount "payload-processing-e2e-ipp-a-e1d376" not found 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls Add eth0 [10.133.0.196/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz Add eth0 [10.133.0.197/23] from ovn-kubernetes 29m Normal Created pod/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz Container created 29m Normal Started pod/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz Container started 29m Normal Pulled pod/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-75c17d-openshift-default Scaled down replica set e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf from 2 to 1 29m Normal Killing pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-75c17d-openshift-default New size: 1; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf Deleted pod: e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-wdsp5 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-c3bbba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-c3bbba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-203209-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-203209-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-203209-65fb56fbdb Error creating: pods "payload-processing-e2e-ipp-a-203209-65fb56fbdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-203209: serviceaccount "payload-processing-e2e-ipp-a-203209" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc Created pod: payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-203209 Scaled up replica set payload-processing-e2e-ipp-a-203209-65fb56fbdb from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-203209-65fb56fbdb Created pod: payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc Error creating: pods "payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-203209: serviceaccount "payload-processing-e2e-ipp-a-203209" not found 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-203209 Scaled up replica set payload-pre-processing-e2e-ipp-a-203209-76878dfcdc from 0 to 1 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn Container created 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn Container started 29m Normal Pulled pod/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b Add eth0 [10.133.0.199/23] from ovn-kubernetes 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn Add eth0 [10.133.0.198/23] from ovn-kubernetes 29m Normal Started pod/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b Container started 29m Normal Created pod/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b Container created 29m Normal ScalingReplicaSet deployment/e2e-ipp-b-e60e16-openshift-default Scaled up replica set e2e-ipp-b-e60e16-openshift-default-765bfc6d7f from 0 to 1 29m Normal AddedInterface pod/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 Add eth0 [10.132.1.1/23] from ovn-kubernetes 29m Normal SuccessfulCreate replicaset/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f Created pod: e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 29m Normal AddedLabel gateway/e2e-ipp-b-e60e16 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-e60e16 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-e60e16 Scaled up replica set payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb Created pod: payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb Error creating: pods "payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-e60e16: serviceaccount "payload-processing-e2e-ipp-b-e60e16" not found 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9 Error creating: pods "payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-e60e16: serviceaccount "payload-processing-e2e-ipp-b-e60e16" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-e60e16 Scaled up replica set payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9 from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9 Created pod: payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5 Container started 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq Add eth0 [10.133.0.201/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5 Add eth0 [10.133.0.200/23] from ovn-kubernetes 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq Container created 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5 Container created 28m Normal Started pod/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Killing pod/payload-processing-e2e-ipp-a-cdcf9d-6bcd7474d-2dqn9 Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-cdcf9d-59dd69766c-dfdn8 Stopping container payload-pre-processing 28m Normal Killing pod/e2e-ipp-a-cdcf9d-openshift-default-97867d6c7-bnjmx Stopping container istio-proxy 28m Normal Killing pod/payload-pre-processing-e2e-ipp-b-c3bbba-59fd85f54c-kd7pl Stopping container payload-pre-processing 28m Normal AddedLabel gateway/e2e-ipp-a-8964ac Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-8964ac 28m Normal Killing pod/payload-processing-e2e-ipp-b-c3bbba-76456ccc86-d86kz Stopping container payload-processing 28m Normal AddedInterface pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 Add eth0 [10.133.0.202/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-8964ac-openshift-default-69c4495c4f Created pod: e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-8964ac-openshift-default Scaled up replica set e2e-ipp-a-8964ac-openshift-default-69c4495c4f from 0 to 1 28m Normal Created pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Container created 28m Normal Started pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Container started 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-8964ac Scaled up replica set payload-processing-e2e-ipp-a-8964ac-976f957fd from 0 to 1 28m Normal Pulled pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f Error creating: pods "payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-8964ac: serviceaccount "payload-processing-e2e-ipp-a-8964ac" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f Created pod: payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-jx66m 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-8964ac Scaled up replica set payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-8964ac-976f957fd Created pod: payload-processing-e2e-ipp-a-8964ac-976f957fd-vhvrt 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-8964ac-976f957fd Error creating: pods "payload-processing-e2e-ipp-a-8964ac-976f957fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-8964ac: serviceaccount "payload-processing-e2e-ipp-a-8964ac" not found 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-jx66m Container created 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-jx66m Add eth0 [10.134.0.242/23] from ovn-kubernetes 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-jx66m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-processing-e2e-ipp-a-8964ac-976f957fd-vhvrt Container started 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-jx66m Container started 28m Normal Created pod/payload-processing-e2e-ipp-a-8964ac-976f957fd-vhvrt Container created 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-8964ac-976f957fd-vhvrt Add eth0 [10.134.0.243/23] from ovn-kubernetes 28m Normal Pulled pod/payload-processing-e2e-ipp-a-8964ac-976f957fd-vhvrt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Killing pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-ipp-b-c3bbba-openshift-default-7d9c98b86c-lp5qs Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused 28m Warning ProbeError pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused... 28m Normal Killing pod/payload-processing-e2e-ipp-a-e1d376-7bc6fb5cb8-vtqvz Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e1d376-6fdd9bcc79-m48ls Stopping container payload-pre-processing 28m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx Stopping container payload-processing 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e1d376-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e1d376-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Killing pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Stopping container istio-proxy 27m Normal Killing pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-4z5w4 Stopping container istio-proxy 27m Warning Unhealthy pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-shared-a-w2-a4782a-openshift-default-948cb9897-xz9rw Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-a4782a-5c8dcfdb8c-pccqc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-a4782a-fd478b978-8rkjx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf Stopping container payload-pre-processing 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e1d376 Scaled up replica set payload-processing-e2e-ipp-a-e1d376-6b45d5bf8 from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e1d376 Scaled up replica set payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67 from 0 to 1 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67 Error creating: pods "payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e1d376: serviceaccount "payload-processing-e2e-ipp-a-e1d376" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67 Created pod: payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8 Error creating: pods "payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e1d376: serviceaccount "payload-processing-e2e-ipp-a-e1d376" not found 27m Normal Created pod/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6 Container created 27m Normal Pulled pod/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2 Add eth0 [10.133.0.204/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Started pod/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6 Container started 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6 Add eth0 [10.133.0.205/23] from ovn-kubernetes 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2 Container started 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8 Created pod: payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2 Container created 27m Normal Created pod/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 Container created 27m Normal Pulled pod/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 Container started 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-e60e16-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-e60e16-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning Unhealthy pod/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 Startup probe failed: Get "http://10.132.1.1:15021/healthz/ready": dial tcp 10.132.1.1:15021: connect: connection refused 27m Warning ProbeError pod/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 Startup probe error: Get "http://10.132.1.1:15021/healthz/ready": dial tcp 10.132.1.1:15021: connect: connection refused... 27m Normal Killing pod/payload-pre-processing-e2e-ipp-b-e60e16-bf7bb78c9-vzxt5 Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-ipp-b-e60e16-6dcbbdb4fb-lhmcq Stopping container payload-processing 27m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-75c17d-5f685b8b95-vvxxf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-75c17d-895b5cd48-gwvpr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f Created pod: e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 27m Normal AddedLabel gateway/e2e-ipp-b-f9a8ed Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-f9a8ed 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-f9a8ed-openshift-default Scaled up replica set e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f from 0 to 1 27m Normal AddedInterface pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 Add eth0 [10.133.0.206/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f9a8ed Scaled up replica set payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899 from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f9a8ed Scaled up replica set payload-processing-e2e-ipp-b-f9a8ed-55b66f5548 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548 Created pod: payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-wr2lz 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548 Error creating: pods "payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed: serviceaccount "payload-processing-e2e-ipp-b-f9a8ed" not found 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899 Error creating: pods "payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed: serviceaccount "payload-processing-e2e-ipp-b-f9a8ed" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899 Created pod: payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-v7qrr 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-v7qrr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-v7qrr Add eth0 [10.134.0.245/23] from ovn-kubernetes 27m Normal Started pod/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-wr2lz Container started 27m Normal Created pod/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-wr2lz Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-v7qrr Container started 27m Normal Pulled pod/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-wr2lz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-wr2lz Add eth0 [10.132.1.9/23] from ovn-kubernetes 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-v7qrr Container created 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-e60e16-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-e60e16-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Killing pod/e2e-shared-b-w2-75c17d-openshift-default-7ff59874cf-tz2f8 Stopping container istio-proxy 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f Created pod: e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh 27m Normal AddedLabel gateway/e2e-ipp-a-22d912 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-22d912 27m Normal ScalingReplicaSet deployment/e2e-ipp-a-22d912-openshift-default Scaled up replica set e2e-ipp-a-22d912-openshift-default-6ff4b55d7f from 0 to 1 26m Normal AddedInterface pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Add eth0 [10.133.0.207/23] from ovn-kubernetes 26m Normal Killing pod/payload-processing-e2e-ipp-a-8964ac-976f957fd-vhvrt Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-8964ac-5f55c6c75f-jx66m Stopping container payload-pre-processing 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-22d912 Scaled up replica set payload-processing-e2e-ipp-a-22d912-5989fd8c7b from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-22d912-5989fd8c7b Created pod: payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-22d912-5989fd8c7b Error creating: pods "payload-processing-e2e-ipp-a-22d912-5989fd8c7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-22d912: serviceaccount "payload-processing-e2e-ipp-a-22d912" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc Created pod: payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-22d912 Scaled up replica set payload-pre-processing-e2e-ipp-a-22d912-568fb578bc from 0 to 1 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc Error creating: pods "payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-22d912: serviceaccount "payload-processing-e2e-ipp-a-22d912" not found 26m Normal Created pod/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69 Container created 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk Add eth0 [10.133.0.209/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk Container started 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk Container created 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69 Add eth0 [10.133.0.208/23] from ovn-kubernetes 26m Normal Pulled pod/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Started pod/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69 Container started 26m Normal Killing pod/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm Stopping container payload-processing 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-e60e16 Scaled up replica set payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6 Error creating: pods "payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-e60e16: serviceaccount "payload-processing-e2e-ipp-b-e60e16" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6 Created pod: payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-e60e16 Scaled up replica set payload-pre-processing-e2e-ipp-b-e60e16-785ff85447 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447 Created pod: payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-s94wx 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447 Error creating: pods "payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-e60e16: serviceaccount "payload-processing-e2e-ipp-b-e60e16" not found 26m Normal Pulled pod/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-s94wx Container created 26m Normal Created pod/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg Container created 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg Add eth0 [10.133.0.210/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-s94wx Container started 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-s94wx Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-s94wx Add eth0 [10.134.0.246/23] from ovn-kubernetes 26m Normal Started pod/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg Container started 26m Normal Pulled pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 Container created 26m Normal Started pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 Container started 26m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-2cda44-7b68745cf-vp9kq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-8964ac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-8964ac-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning Unhealthy pod/payload-processing-e2e-worker-w2-2cda44-76f8dccbf4-52gmm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 Startup probe failed: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 Startup probe error: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused... 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6 Error creating: pods "payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-8964ac: serviceaccount "payload-processing-e2e-ipp-a-8964ac" not found 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9 Error creating: pods "payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-8964ac: serviceaccount "payload-processing-e2e-ipp-a-8964ac" not found 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-8964ac Scaled up replica set payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-8964ac Scaled up replica set payload-processing-e2e-ipp-a-8964ac-5557f4d8c6 from 0 to 1 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-ll5l7 Container created 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-ll5l7 Container started 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6 Created pod: payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2 26m Normal Pulled pod/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-ll5l7 Add eth0 [10.134.0.249/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-ll5l7 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2 Add eth0 [10.133.0.211/23] from ovn-kubernetes 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9 Created pod: payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-ll5l7 26m Normal Created pod/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2 Container created 26m Normal Started pod/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2 Container started 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-8964ac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-8964ac-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-e136b8-openshift-default-84c5d67df9 Created pod: e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c 26m Normal AddedInterface pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Add eth0 [10.132.1.14/23] from ovn-kubernetes 26m Normal AddedLabel gateway/e2e-ipp-b-e136b8 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-e136b8 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-e136b8-openshift-default Scaled up replica set e2e-ipp-b-e136b8-openshift-default-84c5d67df9 from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-e136b8-c8c99b868 Error creating: pods "payload-processing-e2e-ipp-b-e136b8-c8c99b868-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-e136b8: serviceaccount "payload-processing-e2e-ipp-b-e136b8" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-e136b8-c8c99b868 Created pod: payload-processing-e2e-ipp-b-e136b8-c8c99b868-b7jb9 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-e136b8 Scaled up replica set payload-processing-e2e-ipp-b-e136b8-c8c99b868 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667 Error creating: pods "payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-e136b8: serviceaccount "payload-processing-e2e-ipp-b-e136b8" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-e136b8 Scaled up replica set payload-pre-processing-e2e-ipp-b-e136b8-76578f5667 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667 Created pod: payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-5tsqf 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-e136b8-c8c99b868-b7jb9 Add eth0 [10.134.0.250/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-b-e136b8-c8c99b868-b7jb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-b-e136b8-c8c99b868-b7jb9 Container created 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-5tsqf Add eth0 [10.134.0.251/23] from ovn-kubernetes 25m Normal Started pod/payload-processing-e2e-ipp-b-e136b8-c8c99b868-b7jb9 Container started 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-5tsqf Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-5tsqf Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-5tsqf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f9a8ed-676c67c899-v7qrr Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-ipp-b-f9a8ed-55b66f5548-wr2lz Stopping container payload-processing 25m Normal Killing pod/e2e-worker-w2-2cda44-openshift-default-9c988549c-mn44v Stopping container istio-proxy 25m Warning ProbeError pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Readiness probe error: Get "http://10.133.0.207:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Warning Unhealthy pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Readiness probe failed: Get "http://10.133.0.207:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Normal Pulled pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Container created 25m Normal Created pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 Container created 25m Normal Started pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 Container started 25m Normal Started pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Container started 25m Normal Pulled pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Startup probe failed: Get "http://10.133.0.207:15021/healthz/ready": dial tcp 10.133.0.207:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Startup probe error: Get "http://10.133.0.207:15021/healthz/ready": dial tcp 10.133.0.207:15021: connect: connection refused... 25m Warning ProbeError pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 Startup probe error: Get "http://10.133.0.206:15021/healthz/ready": dial tcp 10.133.0.206:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 Startup probe failed: Get "http://10.133.0.206:15021/healthz/ready": dial tcp 10.133.0.206:15021: connect: connection refused 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f9a8ed-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f9a8ed-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-95692f-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c Created pod: e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 25m Normal Started pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 Container started 25m Normal Created pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 Container created 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-95692f-openshift-default Scaled up replica set e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c from 2 to 3 25m Normal Pulled pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 Add eth0 [10.134.0.252/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-908469-openshift-default-54f468f759 Created pod: e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-908469-openshift-default Scaled up replica set e2e-shared-a-w6-908469-openshift-default-54f468f759 from 2 to 3 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-908469-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal Pulled pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z Container started 25m Normal Killing pod/payload-processing-e2e-ipp-a-22d912-5989fd8c7b-zzb69 Stopping container payload-processing 25m Normal AddedInterface pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z Add eth0 [10.134.0.253/23] from ovn-kubernetes 25m Normal Created pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z Container created 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-22d912-568fb578bc-n99gk Stopping container payload-pre-processing 25m Warning Unhealthy pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 Startup probe failed: Get "http://10.134.0.252:15021/healthz/ready": dial tcp 10.134.0.252:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 Startup probe error: Get "http://10.134.0.252:15021/healthz/ready": dial tcp 10.134.0.252:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z Startup probe failed: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z Startup probe error: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused... 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f9a8ed-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f9a8ed-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f9a8ed Scaled up replica set payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f9a8ed Scaled up replica set payload-processing-e2e-ipp-b-f9a8ed-db9589fcd from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f Created pod: payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f Error creating: pods "payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed: serviceaccount "payload-processing-e2e-ipp-b-f9a8ed" not found 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd Error creating: pods "payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f9a8ed: serviceaccount "payload-processing-e2e-ipp-b-f9a8ed" not found 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs Add eth0 [10.133.0.213/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd Created pod: payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2 Add eth0 [10.133.0.212/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2 Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2 Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs Container created 25m Normal Started pod/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs Container started 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-22d912-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-22d912-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal AddedLabel gateway/e2e-ipp-a-be3ada Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-be3ada 24m Normal ScalingReplicaSet deployment/e2e-ipp-a-be3ada-openshift-default Scaled up replica set e2e-ipp-a-be3ada-openshift-default-6878f56887 from 0 to 1 24m Normal SuccessfulCreate replicaset/e2e-ipp-a-be3ada-openshift-default-6878f56887 Created pod: e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 24m Normal AddedInterface pod/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 Add eth0 [10.132.1.20/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-22d912 Scaled up replica set payload-pre-processing-e2e-ipp-a-22d912-68799467dc from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-22d912 Scaled up replica set payload-processing-e2e-ipp-a-22d912-78c4446b66 from 0 to 1 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555 Error creating: pods "payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-be3ada: serviceaccount "payload-processing-e2e-ipp-a-be3ada" not found 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-be3ada Scaled up replica set payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555 from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-22d912-78c4446b66 Error creating: pods "payload-processing-e2e-ipp-a-22d912-78c4446b66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-22d912: serviceaccount "payload-processing-e2e-ipp-a-22d912" not found 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-22d912-68799467dc Created pod: payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-be3ada-76ffbdb68 Created pod: payload-processing-e2e-ipp-a-be3ada-76ffbdb68-f8ntj 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-be3ada-76ffbdb68 Error creating: pods "payload-processing-e2e-ipp-a-be3ada-76ffbdb68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-be3ada: serviceaccount "payload-processing-e2e-ipp-a-be3ada" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-be3ada Scaled up replica set payload-processing-e2e-ipp-a-be3ada-76ffbdb68 from 0 to 1 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-22d912-68799467dc Error creating: pods "payload-pre-processing-e2e-ipp-a-22d912-68799467dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-22d912: serviceaccount "payload-processing-e2e-ipp-a-22d912" not found 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-22d912-78c4446b66 Created pod: payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555 Created pod: payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-6sn9p 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-6sn9p Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r Container created 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r Container started 24m Normal Created pod/payload-processing-e2e-ipp-a-be3ada-76ffbdb68-f8ntj Container created 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-be3ada-76ffbdb68-f8ntj Add eth0 [10.132.1.22/23] from ovn-kubernetes 24m Normal Started pod/payload-processing-e2e-ipp-a-be3ada-76ffbdb68-f8ntj Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-6sn9p Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-6sn9p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-6sn9p Add eth0 [10.134.0.254/23] from ovn-kubernetes 24m Normal Created pod/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Pulled pod/payload-processing-e2e-ipp-a-be3ada-76ffbdb68-f8ntj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm Container started 24m Normal Pulled pod/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm Add eth0 [10.133.0.214/23] from ovn-kubernetes 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r Add eth0 [10.133.0.215/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-f8137c-openshift-default Scaled up replica set e2e-ipp-b-f8137c-openshift-default-7fdc8fc477 from 0 to 1 24m Normal AddedLabel gateway/e2e-ipp-b-f8137c Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-f8137c 24m Normal AddedInterface pod/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t Add eth0 [10.132.1.23/23] from ovn-kubernetes 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477 Created pod: e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t 24m Warning ProbeError pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Readiness probe error: Get "http://10.132.1.14:15021/healthz/ready": read tcp 10.132.0.2:52266->10.132.1.14:15021: read: connection reset by peer... 24m Warning Unhealthy pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Readiness probe failed: Get "http://10.132.1.14:15021/healthz/ready": read tcp 10.132.0.2:52266->10.132.1.14:15021: read: connection reset by peer 24m Normal Pulled pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Container created 24m Normal Started pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Container started 24m Normal Killing pod/payload-processing-e2e-ipp-b-e136b8-c8c99b868-b7jb9 Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-b-e136b8-76578f5667-5tsqf Stopping container payload-pre-processing 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f8137c Scaled up replica set payload-processing-e2e-ipp-b-f8137c-77958df654 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff Created pod: payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-dlqvt 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-dlqvt Container started 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff Error creating: pods "payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f8137c: serviceaccount "payload-processing-e2e-ipp-b-f8137c" not found 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-dlqvt Add eth0 [10.134.0.255/23] from ovn-kubernetes 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f8137c-77958df654 Error creating: pods "payload-processing-e2e-ipp-b-f8137c-77958df654-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f8137c: serviceaccount "payload-processing-e2e-ipp-b-f8137c" not found 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f8137c-77958df654 Created pod: payload-processing-e2e-ipp-b-f8137c-77958df654-xxgsq 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f8137c Scaled up replica set payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff from 0 to 1 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-dlqvt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f8137c-77958df654-xxgsq Add eth0 [10.132.1.24/23] from ovn-kubernetes 24m Normal Pulled pod/payload-processing-e2e-ipp-b-f8137c-77958df654-xxgsq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-dlqvt Container created 24m Normal Started pod/payload-processing-e2e-ipp-b-f8137c-77958df654-xxgsq Container started 24m Normal Created pod/payload-processing-e2e-ipp-b-f8137c-77958df654-xxgsq Container created 24m Warning ProbeError pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Startup probe error: Get "http://10.132.1.14:15021/healthz/ready": dial tcp 10.132.1.14:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Startup probe failed: Get "http://10.132.1.14:15021/healthz/ready": dial tcp 10.132.1.14:15021: connect: connection refused 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-e136b8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-e136b8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal Killing pod/payload-processing-e2e-ipp-a-e1d376-6b45d5bf8-dmtj6 Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e1d376-5b695cd67-gvht2 Stopping container payload-pre-processing 23m Normal Killing pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Stopping container istio-proxy 23m Warning ProbeError pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-a-e1d376-openshift-default-77d6b898c-cc4qt Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-e136b8 Scaled up replica set payload-processing-e2e-ipp-b-e136b8-98d5b5cb6 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-e136b8 Scaled up replica set payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499 from 0 to 1 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6 Error creating: pods "payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-e136b8: serviceaccount "payload-processing-e2e-ipp-b-e136b8" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6 Created pod: payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh Container created 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499 Created pod: payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh Add eth0 [10.133.0.216/23] from ovn-kubernetes 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh Container started 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4 Add eth0 [10.133.0.217/23] from ovn-kubernetes 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499 Error creating: pods "payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-e136b8: serviceaccount "payload-processing-e2e-ipp-b-e136b8" not found 23m Normal Started pod/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4 Container started 23m Normal Pulled pod/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4 Container created 23m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-f5bef0-openshift-default Scaled up replica set e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5 from 1 to 2 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-f5bef0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5 Created pod: e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl 23m Normal AddedInterface pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Add eth0 [10.133.0.218/23] from ovn-kubernetes 23m Normal Created pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Container created 23m Normal Started pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Container started 23m Normal Pulled pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Warning ProbeError pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Startup probe error: Get "http://10.133.0.218:15021/healthz/ready": dial tcp 10.133.0.218:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Startup probe failed: Get "http://10.133.0.218:15021/healthz/ready": dial tcp 10.133.0.218:15021: connect: connection refused 23m Normal Created pod/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 Container created 23m Normal Pulled pod/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 Container started 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-be3ada-8ddf8c555-6sn9p Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-be3ada-76ffbdb68-f8ntj Stopping container payload-processing 23m Warning ProbeError pod/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 Startup probe error: Get "http://10.132.1.20:15021/healthz/ready": dial tcp 10.132.1.20:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 Startup probe failed: Get "http://10.132.1.20:15021/healthz/ready": dial tcp 10.132.1.20:15021: connect: connection refused 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-be3ada-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-be3ada-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f9a8ed-7c6c79c59f-cnfj2 Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-f9a8ed-db9589fcd-mk8cs Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f8137c-cc58f54ff-dlqvt Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-f8137c-77958df654-xxgsq Stopping container payload-processing 22m Normal Created pod/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t Container created 22m Normal Pulled pod/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Started pod/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t Container started 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-203209-76878dfcdc-q7lrn Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-a-203209-65fb56fbdb-r729b Stopping container payload-processing 22m Normal Killing pod/e2e-ipp-b-f9a8ed-openshift-default-5569d9f89f-v4n29 Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t Startup probe failed: Get "http://10.132.1.23:15021/healthz/ready": dial tcp 10.132.1.23:15021: connect: connection refused 22m Warning ProbeError pod/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t Startup probe error: Get "http://10.132.1.23:15021/healthz/ready": dial tcp 10.132.1.23:15021: connect: connection refused... 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-8964ac-5f69ddf9c9-ll5l7 Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-a-8964ac-5557f4d8c6-8clg2 Stopping container payload-processing 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94 Error creating: pods "payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-be3ada: serviceaccount "payload-processing-e2e-ipp-a-be3ada" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94 Created pod: payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-be3ada Scaled up replica set payload-processing-e2e-ipp-a-be3ada-c5f5cfb94 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c Created pod: payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c Error creating: pods "payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-be3ada: serviceaccount "payload-processing-e2e-ipp-a-be3ada" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-be3ada Scaled up replica set payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c from 0 to 1 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc Container created 22m Normal Started pod/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s Container started 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s Add eth0 [10.133.0.220/23] from ovn-kubernetes 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc Add eth0 [10.133.0.219/23] from ovn-kubernetes 22m Normal Pulled pod/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-be3ada-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-be3ada-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal Killing pod/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq Stopping container payload-processing 22m Normal Killing pod/e2e-ipp-a-8964ac-openshift-default-69c4495c4f-5l5w5 Stopping container istio-proxy 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f Stopping container payload-pre-processing 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f8137c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f8137c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-480d41-openshift-default-68f6bb9c47 Created pod: e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf 22m Normal AddedLabel gateway/e2e-ipp-b-480d41 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-480d41 22m Normal ScalingReplicaSet deployment/e2e-ipp-b-480d41-openshift-default Scaled up replica set e2e-ipp-b-480d41-openshift-default-68f6bb9c47 from 0 to 1 22m Normal AddedInterface pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf Add eth0 [10.133.0.221/23] from ovn-kubernetes 22m Normal Killing pod/payload-processing-e2e-ipp-b-e136b8-98d5b5cb6-cdbv4 Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-e136b8-7c7c45c499-tnnlh Stopping container payload-pre-processing 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc Error creating: pods "payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-480d41: serviceaccount "payload-processing-e2e-ipp-b-480d41" not found 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-480d41 Scaled up replica set payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-480d41 Scaled up replica set payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8 Error creating: pods "payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-480d41: serviceaccount "payload-processing-e2e-ipp-b-480d41" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8 Created pod: payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq Add eth0 [10.133.0.223/23] from ovn-kubernetes 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f8137c-746969ff88 Error creating: pods "payload-processing-e2e-ipp-b-f8137c-746969ff88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f8137c: serviceaccount "payload-processing-e2e-ipp-b-f8137c" not found 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf Error creating: pods "payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f8137c: serviceaccount "payload-processing-e2e-ipp-b-f8137c" not found 22m Normal Started pod/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc Created pod: payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq 22m Normal Created pod/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq Container created 22m Normal Pulled pod/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq Add eth0 [10.133.0.222/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f8137c-746969ff88 Created pod: payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f8137c Scaled up replica set payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f8137c Scaled up replica set payload-processing-e2e-ipp-b-f8137c-746969ff88 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf Created pod: payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn Add eth0 [10.133.0.224/23] from ovn-kubernetes 22m Normal Created pod/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54 Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn Container created 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54 Add eth0 [10.133.0.225/23] from ovn-kubernetes 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq Container started 22m Normal Pulled pod/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Started pod/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54 Container started 22m Normal Killing pod/e2e-shared-a-w3-ad73a4-openshift-default-5bb7578f85-n44tv Stopping container istio-proxy 22m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-ad73a4-5b685558f8-npfkq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-ad73a4-77f578f55-vhv5f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/e2e-ipp-b-e136b8-openshift-default-84c5d67df9-5fs2c Stopping container istio-proxy 22m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-70d3a9-768b7dbd8c-znkvj Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 Stopping container payload-processing 22m Normal Killing pod/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx Stopping container payload-pre-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb Stopping container payload-processing 22m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-kzl4j 22m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 22m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 21m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-kzl4j Container created 21m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-kzl4j Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 21m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-kzl4j Container started 21m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-kzl4j Add eth0 [10.134.1.2/23] from ovn-kubernetes 21m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-kzl4j Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-kzl4j Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Killing pod/e2e-shared-b-w3-70d3a9-openshift-default-6cb8bc6bf5-qnrdp Stopping container istio-proxy 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-889195-openshift-default-5b79476475 Created pod: e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-889195-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-889195-openshift-default Scaled up replica set e2e-shared-b-w1-889195-openshift-default-5b79476475 from 1 to 2 21m Normal Started pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Container started 21m Normal Created pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Container created 21m Normal AddedInterface pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Add eth0 [10.132.1.42/23] from ovn-kubernetes 21m Normal Pulled pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Warning ProbeError pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Startup probe error: Get "http://10.132.1.42:15021/healthz/ready": dial tcp 10.132.1.42:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Startup probe failed: Get "http://10.132.1.42:15021/healthz/ready": dial tcp 10.132.1.42:15021: connect: connection refused 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-f5bef0-55dd7bcb78-8v6lv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-70d3a9-676bf9fd96-wt7d5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-f5bef0-6898c5fb7-kf5jx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 Stopping container payload-processing 21m Normal Killing pod/e2e-ipp-a-203209-openshift-default-6f6fdc6799-bkx44 Stopping container istio-proxy 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-66cc54-96bc8fd5d-fnsvb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-66cc54-6cdb985879-scftv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Stopping container istio-proxy 21m Warning ProbeError pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-shared-a-w0-66cc54-openshift-default-745899c56f-sgmcm Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Killing pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-zvm8j Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-shared-a-w1-f5bef0-openshift-default-f946cb9b5-hfjcl Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-79477e-7b8bf4694c-vqb4z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-worker-w3-79477e-6dbcd4f6f4-lqpn2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-a-22d912-78c4446b66-mpfmm Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-22d912-68799467dc-jsr4r Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn Stopping container payload-pre-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-e60e16-785ff85447-s94wx Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-e60e16-84f7f6f4b6-kz7rg Stopping container payload-processing 21m Normal Killing pod/e2e-ipp-a-22d912-openshift-default-6ff4b55d7f-27smh Stopping container istio-proxy 21m Normal Started pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf Container started 21m Normal Created pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf Container created 21m Normal Pulled pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Killing pod/e2e-ipp-b-e60e16-openshift-default-765bfc6d7f-slk56 Stopping container istio-proxy 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-480d41-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-480d41-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-9cd83f-86c464f86c-l45bq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-9cd83f-7679545f6d-6rw6h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning ProbeError pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf Startup probe error: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf Startup probe failed: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused 21m Normal Killing pod/payload-processing-e2e-ipp-b-480d41-7d9ff5f6b8-4gbzq Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-480d41-85764f7fdc-x8gsq Stopping container payload-pre-processing 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-889195-66b9c594bd-tpws9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 Stopping container payload-pre-processing 20m Normal Killing pod/e2e-shared-b-w0-9cd83f-openshift-default-776599bcdc-fdst5 Stopping container istio-proxy 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-889195-5dcfc78767-sldjn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning ProbeError pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-7v2rj Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w1-889195-openshift-default-5b79476475-p84wh Stopping container istio-proxy 20m Normal Killing pod/payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db Stopping container payload-pre-processing 20m Normal Killing pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-worker-w3-79477e-openshift-default-55bb567445-m595f Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-480d41-74f757d7bb Created pod: payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-480d41 Scaled up replica set payload-pre-processing-e2e-ipp-b-480d41-685556dd6c from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c Created pod: payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-480d41 Scaled up replica set payload-processing-e2e-ipp-b-480d41-74f757d7bb from 0 to 1 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c Error creating: pods "payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-480d41: serviceaccount "payload-processing-e2e-ipp-b-480d41" not found 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-480d41-74f757d7bb Error creating: pods "payload-processing-e2e-ipp-b-480d41-74f757d7bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-480d41: serviceaccount "payload-processing-e2e-ipp-b-480d41" not found 20m Normal Created pod/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z Container created 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Killing pod/payload-processing-e2e-ipp-b-f8137c-746969ff88-jmw54 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f8137c-56bf595fcf-kwmdn Stopping container payload-pre-processing 20m Normal Started pod/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z Container started 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4 Container created 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4 Container started 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4 Add eth0 [10.133.0.227/23] from ovn-kubernetes 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z Add eth0 [10.133.0.228/23] from ovn-kubernetes 20m Normal Pulled pod/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Killing pod/payload-pre-processing-e2e-worker-w1-56ac74-868c7f7767-xhxqs Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-worker-w1-56ac74-96bdd884-tp8pc Stopping container payload-processing 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-e11bef-847d99c87b-ksqtm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-e11bef-85f69d968c-p8j94 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-a-w4-e11bef-openshift-default-689d85bc75-kvntq Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-b-f8137c-openshift-default-7fdc8fc477-nlf5t Stopping container istio-proxy 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-58fea9-dc57fd4c9-jk9db Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-worker-w0-58fea9-9984455b6-brp87 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-480d41-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-480d41-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 Stopping container payload-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc Stopping container payload-pre-processing 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-c46397-68d7d9d99d-gcqk5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-c46397-7b8fc778dc-zjrpq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-worker-w1-56ac74-openshift-default-859dc8d675-zmzrj Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/e2e-worker-w0-58fea9-openshift-default-7b68988877-ss9t4 Stopping container istio-proxy 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-55c5e0-849756dbfb-wfvzq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-55c5e0-59cdd46bf9-mz6bc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-shared-b-w4-c46397-openshift-default-6cbb7f7fb9-22mbn Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc Readiness probe failed: dial tcp 10.133.0.219:9004: connect: connection refused 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-be3ada-57f9fc984c-xs9tc Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Stopping container istio-proxy 19m Normal Killing pod/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s Stopping container payload-processing 19m Warning Unhealthy pod/payload-processing-e2e-ipp-a-be3ada-c5f5cfb94-vpc2s Readiness probe failed: dial tcp 10.133.0.220:9004: connect: connection refused 19m Normal Killing pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z Stopping container istio-proxy 19m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-908469-openshift-default Scaled down replica set e2e-shared-a-w6-908469-openshift-default-54f468f759 from 3 to 2 19m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-908469-openshift-default-54f468f759 Deleted pod: e2e-shared-a-w6-908469-openshift-default-54f468f759-zdv9z 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-908469-openshift-default New size: 2; reason: All metrics below target 19m Warning Unhealthy pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-f4nj7 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning ProbeError pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-a-w5-55c5e0-openshift-default-85bfdf9f8c-fp4mz Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal Killing pod/payload-processing-e2e-worker-w4-839163-5bf99f9954-qlx92 Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft Stopping container payload-pre-processing 19m Normal Killing pod/e2e-ipp-a-be3ada-openshift-default-6878f56887-xnt45 Stopping container istio-proxy 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q Stopping container payload-processing 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-95692f-openshift-default New size: 2; reason: All metrics below target 19m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-95692f-openshift-default Scaled down replica set e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c from 3 to 2 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c Deleted pod: e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 19m Normal Killing pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-c8w82 Stopping container istio-proxy 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-480d41-685556dd6c-t7xc4 Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-480d41-74f757d7bb-7gb7z Stopping container payload-processing 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-839163-6bb9bf88c-hkdft Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-shared-b-w5-da52fd-openshift-default-6bbb549bd7-7bgsf Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-worker-w4-839163-openshift-default-ff4d9b8f-rjmgg Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-da52fd-7b88fccc79-btv5h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-da52fd-8567bc8548-6nh6q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-ipp-b-480d41-openshift-default-68f6bb9c47-wtgrf Stopping container istio-proxy 19m Normal Killing pod/payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 Stopping container payload-pre-processing 18m Warning ProbeError pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Readiness probe failed: HTTP probe failed with statuscode: 503 18m Normal Killing pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-nxmmg Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-a-w6-908469-openshift-default-54f468f759-gj7s9 Stopping container istio-proxy 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-43c070-6f8cb67579-pp85j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-worker-w5-43c070-78859bb9c8-cnl96 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-908469-c575ddf9f-ll8n2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-908469-994779fdd-55mn7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w Stopping container payload-processing 18m Normal Killing pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-worker-w5-43c070-openshift-default-857c6bc8df-b8s5g Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-gtphm Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w6-95692f-openshift-default-59b6f8c66c-nj5g9 Stopping container istio-proxy 18m Normal Killing pod/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h Stopping container payload-processing 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-95692f-749d9cd959-9jm9p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-95692f-7697cf9c9b-pvc4w Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-ed088d-5c4d48b955-p2s5j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-processing-e2e-worker-w6-ed088d-dfb468776-kmn7h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Stopping container istio-proxy 17m Warning ProbeError pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-worker-w6-ed088d-openshift-default-6d4f744f5-wdzbz Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-h2mcv 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 1 to 2 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 17m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv Container created 17m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv Add eth0 [10.133.0.231/23] from ovn-kubernetes 17m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv Container started 17m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv Startup probe error: Get "http://10.133.0.231:15021/healthz/ready": dial tcp 10.133.0.231:15021: connect: connection refused... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-h2mcv Startup probe failed: Get "http://10.133.0.231:15021/healthz/ready": dial tcp 10.133.0.231:15021: connect: connection refused 16m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 16m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-kzl4j 16m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-kzl4j Stopping container discovery 16m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-p66fx 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-nkgpq 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 2 to 4 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 16m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-nkgpq Container created 16m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx Container started 16m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx Container created 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-nkgpq Add eth0 [10.134.1.8/23] from ovn-kubernetes 16m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-nkgpq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx Add eth0 [10.133.0.232/23] from ovn-kubernetes 16m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-nkgpq Container started 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx Startup probe failed: Get "http://10.133.0.232:15021/healthz/ready": dial tcp 10.133.0.232:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-nkgpq Startup probe error: Get "http://10.134.1.8:15021/healthz/ready": dial tcp 10.134.1.8:15021: connect: connection refused... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-p66fx Startup probe error: Get "http://10.133.0.232:15021/healthz/ready": dial tcp 10.133.0.232:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-nkgpq Startup probe failed: Get "http://10.134.1.8:15021/healthz/ready": dial tcp 10.134.1.8:15021: connect: connection refused 15m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 4 to 5 15m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: 15m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Container created 15m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Container started 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Add eth0 [10.132.1.70/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-rh5rw 15m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Startup probe error: Get "http://10.132.1.70:15021/healthz/ready": dial tcp 10.132.1.70:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Startup probe failed: Get "http://10.132.1.70:15021/healthz/ready": dial tcp 10.132.1.70:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Startup probe error: Get "http://10.132.1.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Startup probe failed: Get "http://10.132.1.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Container started 15m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Container started 15m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Container started 15m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Container created 15m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-bz96z 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-bpsh4 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-pnzz7 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-xgns5 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Add eth0 [10.132.1.71/23] from ovn-kubernetes 15m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Add eth0 [10.133.0.233/23] from ovn-kubernetes 15m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 5 to 9 15m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Container created 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Add eth0 [10.134.1.11/23] from ovn-kubernetes 15m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 15m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Container started 15m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Container created 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Add eth0 [10.134.1.12/23] from ovn-kubernetes 15m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Container created 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Startup probe failed: Get "http://10.132.1.71:15021/healthz/ready": dial tcp 10.132.1.71:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Startup probe error: Get "http://10.132.1.71:15021/healthz/ready": dial tcp 10.132.1.71:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Startup probe failed: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Startup probe error: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused... 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Startup probe error: Get "http://10.134.1.12:15021/healthz/ready": dial tcp 10.134.1.12:15021: connect: connection refused... 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Startup probe error: Get "http://10.134.1.11:15021/healthz/ready": dial tcp 10.134.1.11:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Startup probe failed: Get "http://10.134.1.11:15021/healthz/ready": dial tcp 10.134.1.11:15021: connect: connection refused 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Startup probe failed: Get "http://10.134.1.12:15021/healthz/ready": dial tcp 10.134.1.12:15021: connect: connection refused 15m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 15m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-57f78bcd55 from 9 to 10 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-57f78bcd55 Created pod: maas-default-gateway-openshift-default-57f78bcd55-nb8vx 15m Normal Started pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Container started 15m Normal Created pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Container created 15m Normal Pulled pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Add eth0 [10.133.0.234/23] from ovn-kubernetes 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Startup probe error: Get "http://10.133.0.234:15021/healthz/ready": dial tcp 10.133.0.234:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Startup probe failed: Get "http://10.133.0.234:15021/healthz/ready": dial tcp 10.133.0.234:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Readiness probe error: Get "http://10.132.1.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-bpsh4 Readiness probe failed: Get "http://10.132.1.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Startup probe failed: Get "http://10.133.0.234:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Startup probe error: Get "http://10.133.0.234:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Readiness probe failed: Get "http://10.134.1.11:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Readiness probe error: Get "http://10.134.1.12:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-bz96z Readiness probe failed: Get "http://10.134.1.12:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-xgns5 Readiness probe error: Get "http://10.134.1.11:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Readiness probe error: Get "http://10.133.0.233:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-pnzz7 Readiness probe failed: Get "http://10.133.0.233:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Normal AddedLabel gateway/e2e-worker-single-main-d36cd3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-d36cd3 12m Normal ScalingReplicaSet deployment/e2e-worker-single-main-d36cd3-openshift-default Scaled up replica set e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f7 from 0 to 1 12m Normal SuccessfulCreate replicaset/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f7 Created pod: e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf 12m Normal AddedInterface pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Add eth0 [10.133.0.235/23] from ovn-kubernetes 12m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fd6 Error creating: pods "payload-processing-e2e-worker-single-main-d36cd3-757d9c5fd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3: serviceaccount "payload-processing-e2e-worker-single-main-d36cd3" not found 12m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77d5c9f Created pod: payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj 12m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-d36cd3 Scaled up replica set payload-processing-e2e-worker-single-main-d36cd3-757d9c5fd6 from 0 to 1 12m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77d5c9f Error creating: pods "payload-pre-processing-e2e-worker-single-main-d36cd3-bb77d5c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-d36cd3: serviceaccount "payload-processing-e2e-worker-single-main-d36cd3" not found 12m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-d36cd3 Scaled up replica set payload-pre-processing-e2e-worker-single-main-d36cd3-bb77d5c9f from 0 to 1 12m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fd6 Created pod: payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 12m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal Started pod/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 Container started 12m Normal Started pod/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj Container started 12m Normal Created pod/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj Container created 12m Normal Created pod/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 Container created 12m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj Add eth0 [10.134.1.25/23] from ovn-kubernetes 12m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 Add eth0 [10.133.0.236/23] from ovn-kubernetes 12m Normal Pulled pod/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Warning Unhealthy pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Readiness probe failed: Get "http://10.133.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Readiness probe error: Get "http://10.133.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Normal Created pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Container created 11m Normal Pulled pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Started pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Container started 11m Warning ProbeError pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Startup probe error: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused... 11m Warning Unhealthy pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Startup probe failed: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-d36cd3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-d36cd3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Normal Killing pod/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 Stopping container payload-processing 10m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj Stopping container payload-pre-processing 10m Normal Killing pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Stopping container istio-proxy 10m Warning ProbeError pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Readiness probe error: HTTP probe failed with statuscode: 503... 10m Warning Unhealthy pod/e2e-worker-single-main-d36cd3-openshift-default-58fcd9f8f76bttf Readiness probe failed: HTTP probe failed with statuscode: 503 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-d36cd3-bb77dv4ljj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-d36cd3-757d9c5fdfzw54 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m42s Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Readiness probe error: Get "http://10.132.1.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m42s Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-rh5rw Readiness probe failed: Get "http://10.132.1.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m26s Normal AddedLabel gateway/e2e-worker-single-main-9b97ce Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-9b97ce 8m26s Normal SuccessfulCreate replicaset/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9 Created pod: e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg 8m26s Normal ScalingReplicaSet deployment/e2e-worker-single-main-9b97ce-openshift-default Scaled up replica set e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9 from 0 to 1 8m25s Normal AddedInterface pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Add eth0 [10.133.0.237/23] from ovn-kubernetes 8m20s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-9b97ce Scaled up replica set payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9cd from 0 to 1 8m20s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f978c47 Created pod: payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh 8m20s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f978c47 Error creating: pods "payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f978c47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-9b97ce: serviceaccount "payload-processing-e2e-worker-single-main-9b97ce" not found 8m20s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9cd Created pod: payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw 8m20s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9cd Error creating: pods "payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-9b97ce: serviceaccount "payload-processing-e2e-worker-single-main-9b97ce" not found 8m20s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-9b97ce Scaled up replica set payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f978c47 from 0 to 1 8m19s Normal Started pod/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw Container started 8m19s Normal Created pod/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw Container created 8m19s Normal Pulled pod/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m19s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw Add eth0 [10.134.1.32/23] from ovn-kubernetes 8m19s Normal Created pod/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh Container created 8m19s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m19s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh Add eth0 [10.134.1.31/23] from ovn-kubernetes 8m19s Normal Started pod/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh Container started 8m7s Warning ProbeError pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Readiness probe error: Get "http://10.133.0.237:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m7s Warning Unhealthy pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Readiness probe failed: Get "http://10.133.0.237:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m6s Normal Pulled pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m6s Normal Created pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Container created 8m6s Normal Started pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Container started 8m1s Warning Unhealthy pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Startup probe failed: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused 8m1s Warning ProbeError pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Startup probe error: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused... 7m41s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-9b97ce-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m41s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-9b97ce-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m13s Normal Killing pod/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw Stopping container payload-processing 7m13s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh Stopping container payload-pre-processing 7m4s Normal Killing pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Stopping container istio-proxy 6m59s Warning ProbeError pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Readiness probe error: HTTP probe failed with statuscode: 503... 6m59s Warning Unhealthy pod/e2e-worker-single-main-9b97ce-openshift-default-fd877fdb9-zm7kg Readiness probe failed: HTTP probe failed with statuscode: 503 6m58s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m58s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m47s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-9b97ce-56bdc8f9c8jnrw Readiness probe failed: timeout: failed to connect service "10.134.1.32:50052" within 1s: context deadline exceeded 6m47s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-9b97ce-6f9f9lg6fh Readiness probe failed: timeout: failed to connect service "10.134.1.31:50052" within 1s: context deadline exceeded 5m56s Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Readiness probe error: Get "http://10.132.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m56s Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-wxstq Readiness probe failed: Get "http://10.132.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m26s Normal ScalingReplicaSet deployment/e2e-ait-6ad3224d-openshift-default Scaled up replica set e2e-ait-6ad3224d-openshift-default-7bfbcc774f from 0 to 1 5m26s Normal AddedLabel gateway/e2e-ait-6ad3224d Added label istio.io/rev=openshift-gateway to gateway e2e-ait-6ad3224d 5m26s Normal SuccessfulCreate replicaset/e2e-ait-6ad3224d-openshift-default-7bfbcc774f Created pod: e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx 5m25s Normal AddedInterface pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Add eth0 [10.133.0.238/23] from ovn-kubernetes 5m20s Warning FailedCreate replicaset/payload-processing-e2e-ait-6ad3224d-79cfdfb994 Error creating: pods "payload-processing-e2e-ait-6ad3224d-79cfdfb994-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-6ad3224d: serviceaccount "payload-processing-e2e-ait-6ad3224d" not found 5m20s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-6ad3224d Scaled up replica set payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc from 0 to 1 5m20s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-6ad3224d Scaled up replica set payload-processing-e2e-ait-6ad3224d-79cfdfb994 from 0 to 1 5m20s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc Error creating: pods "payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-6ad3224d: serviceaccount "payload-processing-e2e-ait-6ad3224d" not found 5m19s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-6ad3224d-79cfdfb994 Created pod: payload-processing-e2e-ait-6ad3224d-79cfdfb994-rnnj2 5m19s Normal Started pod/payload-processing-e2e-ait-6ad3224d-79cfdfb994-rnnj2 Container started 5m19s Normal Created pod/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-5tf9b Container created 5m19s Normal Pulled pod/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-5tf9b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m19s Normal Started pod/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-5tf9b Container started 5m19s Normal AddedInterface pod/payload-processing-e2e-ait-6ad3224d-79cfdfb994-rnnj2 Add eth0 [10.134.1.42/23] from ovn-kubernetes 5m19s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc Created pod: payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-5tf9b 5m19s Normal Pulled pod/payload-processing-e2e-ait-6ad3224d-79cfdfb994-rnnj2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m19s Normal AddedInterface pod/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-5tf9b Add eth0 [10.134.1.41/23] from ovn-kubernetes 5m19s Normal Created pod/payload-processing-e2e-ait-6ad3224d-79cfdfb994-rnnj2 Container created 5m7s Warning ProbeError pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Readiness probe error: Get "http://10.133.0.238:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m7s Warning Unhealthy pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Readiness probe failed: Get "http://10.133.0.238:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m5s Normal Created pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Container created 5m5s Normal Pulled pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m5s Normal Started pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Container started 4m59s Warning ProbeError pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Startup probe error: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused... 4m59s Warning Unhealthy pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Startup probe failed: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused 4m56s Normal Killing pod/payload-processing-e2e-ait-6ad3224d-79cfdfb994-rnnj2 Stopping container payload-processing 4m56s Normal Killing pod/payload-pre-processing-e2e-ait-6ad3224d-57fbf7b9bc-5tf9b Stopping container payload-pre-processing 4m56s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-6ad3224d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m56s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-6ad3224d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m50s Normal Killing pod/e2e-ait-6ad3224d-openshift-default-7bfbcc774f-v5ngx Stopping container istio-proxy 4m49s Normal ScalingReplicaSet deployment/e2e-authcfg-http-8c3a04-openshift-default Scaled up replica set e2e-authcfg-http-8c3a04-openshift-default-d7758c569 from 0 to 1 4m49s Normal AddedLabel gateway/e2e-authcfg-http-8c3a04 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-8c3a04 4m49s Normal SuccessfulCreate replicaset/e2e-authcfg-http-8c3a04-openshift-default-d7758c569 Created pod: e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx 4m48s Normal AddedInterface pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Add eth0 [10.133.0.240/23] from ovn-kubernetes 4m43s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4 Created pod: payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-skfxn 4m43s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-8c3a04 Scaled up replica set payload-processing-e2e-authcfg-http-8c3a04-594499f4c4 from 0 to 1 4m43s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475 Error creating: pods "payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-8c3a04: serviceaccount "payload-processing-e2e-authcfg-http-8c3a04" not found 4m43s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475 Created pod: payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj 4m43s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4 Error creating: pods "payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-8c3a04: serviceaccount "payload-processing-e2e-authcfg-http-8c3a04" not found 4m43s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-8c3a04 Scaled up replica set payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475 from 0 to 1 4m42s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-skfxn Add eth0 [10.134.1.44/23] from ovn-kubernetes 4m42s Normal Pulled pod/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-skfxn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m42s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m42s Normal Started pod/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj Container started 4m42s Normal Created pod/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj Container created 4m42s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj Add eth0 [10.133.0.241/23] from ovn-kubernetes 4m42s Normal Started pod/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-skfxn Container started 4m42s Normal Created pod/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-skfxn Container created 4m30s Warning ProbeError pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Readiness probe error: Get "http://10.133.0.240:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 4m30s Warning Unhealthy pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Readiness probe failed: Get "http://10.133.0.240:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 4m29s Normal Created pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Container created 4m29s Normal Pulled pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m29s Normal Started pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Container started 4m23s Warning ProbeError pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Startup probe error: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused... 4m23s Warning Unhealthy pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Startup probe failed: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused 4m18s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-8c3a04-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m18s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-8c3a04-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m3s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-8c3a04-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m3s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-8c3a04-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 2m59s Normal Killing pod/payload-processing-e2e-authcfg-http-8c3a04-594499f4c4-skfxn Stopping container payload-processing 2m59s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-8c3a04-6886cb8475-m6qhj Stopping container payload-pre-processing 2m46s Normal Killing pod/e2e-authcfg-http-8c3a04-openshift-default-d7758c569-znmrx Stopping container istio-proxy 2m11s Warning Unhealthy pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Readiness probe failed: Get "http://10.133.0.234:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m11s Warning ProbeError pod/maas-default-gateway-openshift-default-57f78bcd55-nb8vx Readiness probe error: Get "http://10.133.0.234:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...