=== events in openshift-ingress (2026-10-29T23:21:27Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 20m Normal Scheduled pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Successfully assigned openshift-ingress/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f to ip-10-0-134-94.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-n4spc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-n4spc to ip-10-0-139-29.ec2.internal 41m Warning FailedScheduling pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Successfully assigned openshift-ingress/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 to ip-10-0-134-94.ec2.internal 50m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb to ip-10-0-139-29.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-api-a-f76fb5-5d75b85b9d-64hcd Successfully assigned openshift-ingress/payload-processing-e2e-api-a-f76fb5-5d75b85b9d-64hcd to ip-10-0-139-29.ec2.internal 35m Normal Scheduled pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 Successfully assigned openshift-ingress/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd Successfully assigned openshift-ingress/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd to ip-10-0-133-251.ec2.internal 47m Normal Scheduled pod/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv Successfully assigned openshift-ingress/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv to ip-10-0-133-251.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-api-a-1ae775-5b67fb4bfd-2829w Successfully assigned openshift-ingress/payload-processing-e2e-api-a-1ae775-5b67fb4bfd-2829w to ip-10-0-133-251.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd Successfully assigned openshift-ingress/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd to ip-10-0-139-29.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-api-b-433712-7f84f4fdbd-sdj56 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-433712-7f84f4fdbd-sdj56 to ip-10-0-139-29.ec2.internal 20m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-lkrpw Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lkrpw to ip-10-0-134-94.ec2.internal 46m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-st7pn to ip-10-0-134-94.ec2.internal 39m Warning FailedScheduling pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 53m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk to ip-10-0-139-29.ec2.internal 59m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-kg9ds Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-kg9ds to ip-10-0-133-251.ec2.internal 38m Warning FailedScheduling pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 49m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 to ip-10-0-133-251.ec2.internal 38m Normal Scheduled pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 Successfully assigned openshift-ingress/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 to ip-10-0-134-94.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-bsvsw Successfully assigned openshift-ingress/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-bsvsw to ip-10-0-133-251.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-fb88b Successfully assigned openshift-ingress/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-fb88b to ip-10-0-139-29.ec2.internal 43m Normal Scheduled pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Successfully assigned openshift-ingress/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm to ip-10-0-139-29.ec2.internal 46m Normal Scheduled pod/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw Successfully assigned openshift-ingress/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xc4fv to ip-10-0-139-29.ec2.internal 35m Normal Scheduled pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw Successfully assigned openshift-ingress/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw to ip-10-0-133-251.ec2.internal 43m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-z6q2x Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z6q2x to ip-10-0-139-29.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Successfully assigned openshift-ingress/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l to ip-10-0-139-29.ec2.internal 22m Normal Scheduled pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv Successfully assigned openshift-ingress/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv to ip-10-0-139-29.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph Successfully assigned openshift-ingress/payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph to ip-10-0-133-251.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-4ldkz Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-4ldkz to ip-10-0-133-251.ec2.internal 26m Normal Scheduled pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 Successfully assigned openshift-ingress/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 to ip-10-0-133-251.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw to ip-10-0-134-94.ec2.internal 52m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx to ip-10-0-139-29.ec2.internal 44m Normal Scheduled pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Successfully assigned openshift-ingress/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv to ip-10-0-139-29.ec2.internal 39m Warning FailedScheduling pod/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 31m Normal Scheduled pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Successfully assigned openshift-ingress/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv to ip-10-0-139-29.ec2.internal 38m Warning FailedScheduling pod/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt Successfully assigned openshift-ingress/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt to ip-10-0-133-251.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb Successfully assigned openshift-ingress/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb to ip-10-0-134-94.ec2.internal 28m Normal Scheduled pod/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn Successfully assigned openshift-ingress/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn to ip-10-0-133-251.ec2.internal 51m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 to ip-10-0-134-94.ec2.internal 54m Normal Scheduled pod/payload-processing-5f78687d5b-zt6f2 Successfully assigned openshift-ingress/payload-processing-5f78687d5b-zt6f2 to ip-10-0-133-251.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz to ip-10-0-139-29.ec2.internal 41m Warning FailedScheduling pod/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 48m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t to ip-10-0-134-94.ec2.internal 42m Normal Scheduled pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 Successfully assigned openshift-ingress/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 to ip-10-0-133-251.ec2.internal 29m Normal Scheduled pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Successfully assigned openshift-ingress/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx to ip-10-0-134-94.ec2.internal 42m Normal Scheduled pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Successfully assigned openshift-ingress/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf to ip-10-0-134-94.ec2.internal 48m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 to ip-10-0-133-251.ec2.internal 60m Normal Scheduled pod/kube-auth-proxy-b649d648f-h99f8 Successfully assigned openshift-ingress/kube-auth-proxy-b649d648f-h99f8 to ip-10-0-133-251.ec2.internal 41m Warning FailedScheduling pod/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Successfully assigned openshift-ingress/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv to ip-10-0-139-29.ec2.internal 42m Warning FailedScheduling pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Successfully assigned openshift-ingress/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-mvczb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-mvczb to ip-10-0-139-29.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 to ip-10-0-139-29.ec2.internal 60m Normal Scheduled pod/kube-auth-proxy-b649d648f-s7zqw Successfully assigned openshift-ingress/kube-auth-proxy-b649d648f-s7zqw to ip-10-0-139-29.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-derive-591986bb-8559c645b8-92szf Successfully assigned openshift-ingress/payload-processing-e2e-derive-591986bb-8559c645b8-92szf to ip-10-0-139-29.ec2.internal 49m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw to ip-10-0-133-251.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx to ip-10-0-139-29.ec2.internal 21m Normal Scheduled pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 Successfully assigned openshift-ingress/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 to ip-10-0-134-94.ec2.internal 43m Normal Scheduled pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Successfully assigned openshift-ingress/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h to ip-10-0-139-29.ec2.internal 29m Normal Scheduled pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg Successfully assigned openshift-ingress/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg to ip-10-0-134-94.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-172c10-57c45696d8-9697h Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-172c10-57c45696d8-9697h to ip-10-0-139-29.ec2.internal 52m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk to ip-10-0-133-251.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl to ip-10-0-139-29.ec2.internal 34m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 to ip-10-0-133-251.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt to ip-10-0-139-29.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn to ip-10-0-139-29.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln Successfully assigned openshift-ingress/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Successfully assigned openshift-ingress/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 to ip-10-0-139-29.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp to ip-10-0-139-29.ec2.internal 49m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 to ip-10-0-133-251.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq to ip-10-0-133-251.ec2.internal 50m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 to ip-10-0-134-94.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp to ip-10-0-139-29.ec2.internal 8m57s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 Successfully assigned openshift-ingress/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 to ip-10-0-134-94.ec2.internal 34m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg to ip-10-0-139-29.ec2.internal 5m33s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 to ip-10-0-133-251.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 to ip-10-0-133-251.ec2.internal 41m Warning FailedScheduling pod/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl Successfully assigned openshift-ingress/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl to ip-10-0-133-251.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf to ip-10-0-133-251.ec2.internal 32m Normal Scheduled pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn Successfully assigned openshift-ingress/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn to ip-10-0-139-29.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm to ip-10-0-133-251.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs to ip-10-0-133-251.ec2.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 to ip-10-0-139-29.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk to ip-10-0-139-29.ec2.internal 35m Normal Scheduled pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Successfully assigned openshift-ingress/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr to ip-10-0-133-251.ec2.internal 34m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs to ip-10-0-133-251.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 to ip-10-0-139-29.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm to ip-10-0-139-29.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 to ip-10-0-139-29.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb to ip-10-0-139-29.ec2.internal 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g Successfully assigned openshift-ingress/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g to ip-10-0-134-94.ec2.internal 41m Warning FailedScheduling pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 20m Normal Scheduled pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Successfully assigned openshift-ingress/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj to ip-10-0-139-29.ec2.internal 41m Warning FailedScheduling pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn Successfully assigned openshift-ingress/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn to ip-10-0-133-251.ec2.internal 34m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 to ip-10-0-134-94.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 to ip-10-0-134-94.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx to ip-10-0-139-29.ec2.internal 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m Successfully assigned openshift-ingress/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m to ip-10-0-133-251.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn to ip-10-0-133-251.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j to ip-10-0-133-251.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b to ip-10-0-139-29.ec2.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-b-8dd4c8-bbd854d48-b8nkm Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-8dd4c8-bbd854d48-b8nkm to ip-10-0-139-29.ec2.internal 20m Normal Scheduled pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Successfully assigned openshift-ingress/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 to ip-10-0-134-94.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-b-8ebc60-58cbf955c8-7swqv Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-8ebc60-58cbf955c8-7swqv to ip-10-0-139-29.ec2.internal 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh Successfully assigned openshift-ingress/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t to ip-10-0-139-29.ec2.internal 41m Warning FailedScheduling pod/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv to ip-10-0-133-251.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc to ip-10-0-139-29.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw to ip-10-0-139-29.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk to ip-10-0-134-94.ec2.internal 42m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj Successfully assigned openshift-ingress/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj to ip-10-0-133-251.ec2.internal 23m Normal Scheduled pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 Successfully assigned openshift-ingress/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 to ip-10-0-139-29.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp to ip-10-0-134-94.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn to ip-10-0-139-29.ec2.internal 34m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd to ip-10-0-134-94.ec2.internal 49m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr to ip-10-0-134-94.ec2.internal 26m Normal Scheduled pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Successfully assigned openshift-ingress/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b to ip-10-0-133-251.ec2.internal 42m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Successfully assigned openshift-ingress/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr to ip-10-0-133-251.ec2.internal 22m Normal Scheduled pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk Successfully assigned openshift-ingress/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk to ip-10-0-139-29.ec2.internal 45m Normal Scheduled pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Successfully assigned openshift-ingress/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 to ip-10-0-133-251.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 to ip-10-0-134-94.ec2.internal 50m Normal Scheduled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Successfully assigned openshift-ingress/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm to ip-10-0-133-251.ec2.internal 22m Normal Scheduled pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Successfully assigned openshift-ingress/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s to ip-10-0-139-29.ec2.internal 21m Normal Scheduled pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Successfully assigned openshift-ingress/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs to ip-10-0-134-94.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 to ip-10-0-134-94.ec2.internal 49m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl to ip-10-0-134-94.ec2.internal 42m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr Successfully assigned openshift-ingress/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr to ip-10-0-133-251.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-g5fq9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-g5fq9 to ip-10-0-133-251.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s to ip-10-0-134-94.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll to ip-10-0-133-251.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-14b0c3-59db99bc46-x2jcd Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-14b0c3-59db99bc46-x2jcd to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-14b0c3-64f964956d-d45gc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-14b0c3-64f964956d-d45gc to ip-10-0-133-251.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-1dabb8-586496bcdd-hhlk9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-1dabb8-586496bcdd-hhlk9 to ip-10-0-139-29.ec2.internal 35m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft to ip-10-0-134-94.ec2.internal 43m Normal Scheduled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Successfully assigned openshift-ingress/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph to ip-10-0-134-94.ec2.internal 53m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr to ip-10-0-133-251.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-fwpxz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-fwpxz to ip-10-0-134-94.ec2.internal 33m Normal Scheduled pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 Successfully assigned openshift-ingress/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 to ip-10-0-134-94.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-3e2b2c-799768d694-x5lm7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-3e2b2c-799768d694-x5lm7 to ip-10-0-139-29.ec2.internal 51m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn to ip-10-0-133-251.ec2.internal 45m Normal Scheduled pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Successfully assigned openshift-ingress/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll to ip-10-0-139-29.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr to ip-10-0-133-251.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b to ip-10-0-139-29.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc to ip-10-0-139-29.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-a-9ee1fc-54c499f956-hqp5l Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-9ee1fc-54c499f956-hqp5l to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-9ee1fc-5746854446-rqk4w Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-9ee1fc-5746854446-rqk4w to ip-10-0-139-29.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-wcfms Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-wcfms to ip-10-0-133-251.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-hvw9t Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-hvw9t to ip-10-0-133-251.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw to ip-10-0-133-251.ec2.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr to ip-10-0-134-94.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 to ip-10-0-133-251.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-f7a5fe-c864778c6-vw2dk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f7a5fe-c864778c6-vw2dk to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p to ip-10-0-133-251.ec2.internal 48m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 to ip-10-0-139-29.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-0906f4-76b779f4dd-xq6x4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-0906f4-76b779f4dd-xq6x4 to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-0906f4-7d7f495d74-qf86f Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-0906f4-7d7f495d74-qf86f to ip-10-0-133-251.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-09fc89-695597ffd8-bntdb Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-09fc89-695597ffd8-bntdb to ip-10-0-133-251.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-09fc89-847d958946-mcjkf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-09fc89-847d958946-mcjkf to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-dppph Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-dppph to ip-10-0-133-251.ec2.internal 39m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc to ip-10-0-134-94.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-2d286f-79f5f996b4-lk6fc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-2d286f-79f5f996b4-lk6fc to ip-10-0-133-251.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 to ip-10-0-133-251.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj to ip-10-0-133-251.ec2.internal 52m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 to ip-10-0-134-94.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn to ip-10-0-139-29.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-bcb810-75ccffb45d-hjkxs Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-bcb810-75ccffb45d-hjkxs to ip-10-0-133-251.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-b-bcb810-cf5bc884-7dvv6 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-bcb810-cf5bc884-7dvv6 to ip-10-0-134-94.ec2.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-p2rgc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-p2rgc to ip-10-0-133-251.ec2.internal 34m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk to ip-10-0-139-29.ec2.internal 39m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj to ip-10-0-133-251.ec2.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-c74e55-7b8b55996d-zt9mc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-c74e55-7b8b55996d-zt9mc to ip-10-0-133-251.ec2.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-eddb03-598b66b8d8-lc677 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-eddb03-598b66b8d8-lc677 to ip-10-0-133-251.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-eddb03-66bc479ffd-9qht6 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-eddb03-66bc479ffd-9qht6 to ip-10-0-139-29.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-migrate-3593b8ba-66df9dddf4-5rbrv Successfully assigned openshift-ingress/payload-processing-e2e-migrate-3593b8ba-66df9dddf4-5rbrv to ip-10-0-133-251.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Successfully assigned openshift-ingress/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 to ip-10-0-133-251.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 to ip-10-0-133-251.ec2.internal 18m Normal Scheduled pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 Successfully assigned openshift-ingress/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 to ip-10-0-133-251.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc to ip-10-0-133-251.ec2.internal 40m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c to ip-10-0-133-251.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h to ip-10-0-139-29.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Successfully assigned openshift-ingress/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm to ip-10-0-133-251.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt Successfully assigned openshift-ingress/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt to ip-10-0-139-29.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 to ip-10-0-139-29.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 to ip-10-0-134-94.ec2.internal 40m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 to ip-10-0-134-94.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n Successfully assigned openshift-ingress/payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n to ip-10-0-133-251.ec2.internal 23m Normal Scheduled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj Successfully assigned openshift-ingress/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj to ip-10-0-139-29.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj to ip-10-0-139-29.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt to ip-10-0-134-94.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Successfully assigned openshift-ingress/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn to ip-10-0-134-94.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh to ip-10-0-133-251.ec2.internal 40m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk to ip-10-0-133-251.ec2.internal 40m Warning FailedScheduling pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 20m Normal Scheduled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 Successfully assigned openshift-ingress/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 to ip-10-0-139-29.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 Successfully assigned openshift-ingress/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl Successfully assigned openshift-ingress/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl to ip-10-0-139-29.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-l7s7s Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-l7s7s to ip-10-0-133-251.ec2.internal 66m Normal Scheduled pod/router-default-b64c8c7d5-7zxkb Successfully assigned openshift-ingress/router-default-b64c8c7d5-7zxkb to ip-10-0-134-94.ec2.internal 67m Warning FailedScheduling pod/router-default-b64c8c7d5-7zxkb no nodes available to schedule pods 72m Warning FailedScheduling pod/router-default-b64c8c7d5-7zxkb no nodes available to schedule pods 39m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 to ip-10-0-139-29.ec2.internal 48m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc to ip-10-0-134-94.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-n7lx5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-n7lx5 to ip-10-0-139-29.ec2.internal 20m Normal Scheduled pod/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw Successfully assigned openshift-ingress/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m to ip-10-0-133-251.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg to ip-10-0-139-29.ec2.internal 35m Normal Scheduled pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Successfully assigned openshift-ingress/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Successfully assigned openshift-ingress/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 to ip-10-0-133-251.ec2.internal 42m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 to ip-10-0-139-29.ec2.internal 20m Normal Scheduled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q Successfully assigned openshift-ingress/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q to ip-10-0-133-251.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv Successfully assigned openshift-ingress/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv to ip-10-0-139-29.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-276bc Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-276bc to ip-10-0-139-29.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-fr5fg Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-fr5fg to ip-10-0-133-251.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s Successfully assigned openshift-ingress/payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s to ip-10-0-139-29.ec2.internal 30m Normal Scheduled pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk Successfully assigned openshift-ingress/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk to ip-10-0-133-251.ec2.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-eddb03-8576855547-rjdcx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-eddb03-8576855547-rjdcx to ip-10-0-133-251.ec2.internal 39m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 to ip-10-0-133-251.ec2.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-vsvld Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-vsvld to ip-10-0-139-29.ec2.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-c4sv5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-c4sv5 to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb to ip-10-0-139-29.ec2.internal 21m Normal Scheduled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Successfully assigned openshift-ingress/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx to ip-10-0-134-94.ec2.internal 49m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt to ip-10-0-139-29.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-qg64m Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-qg64m to ip-10-0-133-251.ec2.internal 26m Normal Scheduled pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Successfully assigned openshift-ingress/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm to ip-10-0-134-94.ec2.internal 39m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m Successfully assigned openshift-ingress/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m to ip-10-0-133-251.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-m8k5n Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-m8k5n to ip-10-0-133-251.ec2.internal 50m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs to ip-10-0-134-94.ec2.internal 44m Normal Scheduled pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Successfully assigned openshift-ingress/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x to ip-10-0-134-94.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Successfully assigned openshift-ingress/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg to ip-10-0-133-251.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg Successfully assigned openshift-ingress/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg to ip-10-0-139-29.ec2.internal 49m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 to ip-10-0-134-94.ec2.internal 44m Normal Scheduled pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Successfully assigned openshift-ingress/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf to ip-10-0-139-29.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Successfully assigned openshift-ingress/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r to ip-10-0-133-251.ec2.internal 47m Normal Scheduled pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Successfully assigned openshift-ingress/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw to ip-10-0-134-94.ec2.internal 49m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n to ip-10-0-134-94.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-4xkd6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-4xkd6 to ip-10-0-133-251.ec2.internal 53m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 to ip-10-0-139-29.ec2.internal 44m Normal Scheduled pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Successfully assigned openshift-ingress/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 Successfully assigned openshift-ingress/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 to ip-10-0-134-94.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-8tzh7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-8tzh7 to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd to ip-10-0-133-251.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-dr4ch Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-dr4ch to ip-10-0-133-251.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-7b98bb678c-8q5x4 Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-8q5x4 to ip-10-0-139-29.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-csl2b Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-csl2b to ip-10-0-139-29.ec2.internal 21m Normal Scheduled pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 Successfully assigned openshift-ingress/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 to ip-10-0-139-29.ec2.internal 38m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d to ip-10-0-133-251.ec2.internal 38m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-54nxc Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-54nxc to ip-10-0-133-251.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs to ip-10-0-139-29.ec2.internal 50m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j to ip-10-0-139-29.ec2.internal 36m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg to ip-10-0-134-94.ec2.internal 41m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 to ip-10-0-139-29.ec2.internal 41m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-f7rhd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-f7rhd to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k to ip-10-0-139-29.ec2.internal 37m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp to ip-10-0-134-94.ec2.internal 38m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-rf9mm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-rf9mm to ip-10-0-133-251.ec2.internal 22m Normal Scheduled pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Successfully assigned openshift-ingress/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff to ip-10-0-133-251.ec2.internal 38m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm to ip-10-0-134-94.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w5-0d12c5-dc776f966-t282d Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-0d12c5-dc776f966-t282d to ip-10-0-139-29.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw to ip-10-0-134-94.ec2.internal 38m Warning FailedScheduling pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Warning FailedScheduling pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q to ip-10-0-139-29.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-gpwpk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-gpwpk to ip-10-0-133-251.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-dtglw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-dtglw to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Successfully assigned openshift-ingress/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b to ip-10-0-133-251.ec2.internal 37m Warning FailedScheduling pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Warning FailedScheduling pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-zwjhm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-zwjhm to ip-10-0-133-251.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 to ip-10-0-139-29.ec2.internal 38m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-mjvbc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-mjvbc to ip-10-0-139-29.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-vt789 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-vt789 to ip-10-0-133-251.ec2.internal 37m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 to ip-10-0-133-251.ec2.internal 42m Warning FailedScheduling pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 47m Normal Scheduled pod/payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c to ip-10-0-133-251.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p to ip-10-0-133-251.ec2.internal 28m Normal Scheduled pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Successfully assigned openshift-ingress/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j to ip-10-0-139-29.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl to ip-10-0-133-251.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Successfully assigned openshift-ingress/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp to ip-10-0-139-29.ec2.internal 39m Warning FailedScheduling pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l to ip-10-0-134-94.ec2.internal 37m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 49m Normal Scheduled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Successfully assigned openshift-ingress/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt to ip-10-0-139-29.ec2.internal 60m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-67df64d57-pzncp Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-67df64d57-pzncp to ip-10-0-134-94.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-pssgt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-pssgt to ip-10-0-133-251.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-b4tfd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-b4tfd to ip-10-0-133-251.ec2.internal 38m Warning FailedScheduling pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Warning FailedScheduling pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl to ip-10-0-139-29.ec2.internal 37m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 32m Normal Scheduled pod/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp Successfully assigned openshift-ingress/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp to ip-10-0-134-94.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s to ip-10-0-139-29.ec2.internal 38m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 26m Normal Scheduled pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Successfully assigned openshift-ingress/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq to ip-10-0-139-29.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-frxg9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-frxg9 to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n to ip-10-0-139-29.ec2.internal 37m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf to ip-10-0-133-251.ec2.internal 38m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-f54wd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-f54wd to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj to ip-10-0-134-94.ec2.internal 37m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 49m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg to ip-10-0-134-94.ec2.internal 37m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Warning FailedScheduling pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p to ip-10-0-133-251.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d to ip-10-0-134-94.ec2.internal 34m Normal Scheduled pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Successfully assigned openshift-ingress/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw to ip-10-0-133-251.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-hpr6v Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-hpr6v to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 to ip-10-0-133-251.ec2.internal 8m57s Normal Scheduled pod/payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t to ip-10-0-133-251.ec2.internal 5m33s Normal Scheduled pod/payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj to ip-10-0-133-251.ec2.internal 37m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd to ip-10-0-139-29.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc to ip-10-0-133-251.ec2.internal 37m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp to ip-10-0-133-251.ec2.internal 38m Warning FailedScheduling pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-6h728 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-6h728 to ip-10-0-139-29.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp Successfully assigned openshift-ingress/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp to ip-10-0-134-94.ec2.internal 48m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 to ip-10-0-133-251.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb to ip-10-0-139-29.ec2.internal 33m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf to ip-10-0-133-251.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn to ip-10-0-133-251.ec2.internal 36m Normal Scheduled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 Successfully assigned openshift-ingress/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 to ip-10-0-134-94.ec2.internal 44m Normal Scheduled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Successfully assigned openshift-ingress/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x to ip-10-0-139-29.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx to ip-10-0-133-251.ec2.internal 53m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 to ip-10-0-133-251.ec2.internal 41m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 46m Normal Scheduled pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 Successfully assigned openshift-ingress/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 to ip-10-0-139-29.ec2.internal 41m Normal Scheduled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Successfully assigned openshift-ingress/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q to ip-10-0-133-251.ec2.internal 47m Normal Scheduled pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-btd2n Successfully assigned openshift-ingress/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-btd2n to ip-10-0-133-251.ec2.internal 42m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj to ip-10-0-139-29.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz to ip-10-0-139-29.ec2.internal 53m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 to ip-10-0-139-29.ec2.internal 42m Normal Scheduled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp Successfully assigned openshift-ingress/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp to ip-10-0-139-29.ec2.internal 42m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br Successfully assigned openshift-ingress/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br to ip-10-0-139-29.ec2.internal 46m Normal Scheduled pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q Successfully assigned openshift-ingress/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q to ip-10-0-134-94.ec2.internal 48m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq to ip-10-0-133-251.ec2.internal 41m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 49m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf to ip-10-0-134-94.ec2.internal 41m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z to ip-10-0-133-251.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 to ip-10-0-139-29.ec2.internal 41m Normal Scheduled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms Successfully assigned openshift-ingress/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms to ip-10-0-133-251.ec2.internal 29m Normal Scheduled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v Successfully assigned openshift-ingress/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v to ip-10-0-133-251.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-b-433712-69d496844c-sfn6r Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-433712-69d496844c-sfn6r to ip-10-0-133-251.ec2.internal 42m Warning FailedScheduling pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp Successfully assigned openshift-ingress/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp to ip-10-0-139-29.ec2.internal 42m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Successfully assigned openshift-ingress/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 to ip-10-0-139-29.ec2.internal 39m Warning FailedScheduling pod/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 51m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk to ip-10-0-133-251.ec2.internal 38m Warning FailedScheduling pod/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 to ip-10-0-134-94.ec2.internal 42m Normal Scheduled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 Successfully assigned openshift-ingress/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 to ip-10-0-139-29.ec2.internal 48m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc to ip-10-0-139-29.ec2.internal 43m Normal Scheduled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Successfully assigned openshift-ingress/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn to ip-10-0-134-94.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-b-c06e05-6485ff7479-6mkhj Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-c06e05-6485ff7479-6mkhj to ip-10-0-139-29.ec2.internal 17m Normal Scheduled pod/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn Successfully assigned openshift-ingress/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn to ip-10-0-139-29.ec2.internal 51m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x to ip-10-0-139-29.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b to ip-10-0-133-251.ec2.internal 46m Normal Scheduled pod/e2e-derive-591986bb-openshift-default-cb894bd5-5thqz Successfully assigned openshift-ingress/e2e-derive-591986bb-openshift-default-cb894bd5-5thqz to ip-10-0-139-29.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-tx58r Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-tx58r to ip-10-0-139-29.ec2.internal 5m38s Normal Scheduled pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Successfully assigned openshift-ingress/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 to ip-10-0-139-29.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw to ip-10-0-139-29.ec2.internal 24m Normal Scheduled pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Successfully assigned openshift-ingress/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns to ip-10-0-139-29.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 to ip-10-0-139-29.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 to ip-10-0-139-29.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 to ip-10-0-133-251.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk to ip-10-0-133-251.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-9xxxz Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-9xxxz to ip-10-0-133-251.ec2.internal 53m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk to ip-10-0-139-29.ec2.internal 30m Normal Scheduled pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Successfully assigned openshift-ingress/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Successfully assigned openshift-ingress/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l to ip-10-0-133-251.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-b96qm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-b96qm to ip-10-0-134-94.ec2.internal 9m3s Normal Scheduled pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Successfully assigned openshift-ingress/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws to ip-10-0-139-29.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg to ip-10-0-133-251.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-r84hx Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-r84hx to ip-10-0-133-251.ec2.internal 53m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 to ip-10-0-134-94.ec2.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx to ip-10-0-139-29.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq to ip-10-0-139-29.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Successfully assigned openshift-ingress/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x to ip-10-0-139-29.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz to ip-10-0-139-29.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl Successfully assigned openshift-ingress/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl to ip-10-0-134-94.ec2.internal 41m Warning FailedScheduling pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m to ip-10-0-133-251.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv to ip-10-0-133-251.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq to ip-10-0-133-251.ec2.internal 41m Warning FailedScheduling pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 26m Normal Scheduled pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Successfully assigned openshift-ingress/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp to ip-10-0-139-29.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf to ip-10-0-133-251.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Successfully assigned openshift-ingress/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq to ip-10-0-134-94.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-zjwr8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-zjwr8 to ip-10-0-133-251.ec2.internal 72m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-b64c8c7d5 from 0 to 1 72m Normal SuccessfulCreate replicaset/router-default-b64c8c7d5 Created pod: router-default-b64c8c7d5-7zxkb 72m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 72m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 72m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 66m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 66m Warning FailedMount pod/router-default-b64c8c7d5-7zxkb MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 66m Warning FailedMount pod/router-default-b64c8c7d5-7zxkb MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Warning FailedMount pod/router-default-b64c8c7d5-7zxkb MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Normal AddedInterface pod/router-default-b64c8c7d5-7zxkb Add eth0 [10.132.0.9/23] from ovn-kubernetes 66m Normal Pulled pod/router-default-b64c8c7d5-7zxkb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 66m Normal Started pod/router-default-b64c8c7d5-7zxkb Container started 66m Normal Created pod/router-default-b64c8c7d5-7zxkb Container created 60m Normal SuccessfulCreate replicaset/kube-auth-proxy-b649d648f Created pod: kube-auth-proxy-b649d648f-s7zqw 60m Normal AddedInterface pod/kube-auth-proxy-b649d648f-h99f8 Add eth0 [10.134.0.16/23] from ovn-kubernetes 60m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-b649d648f from 0 to 2 60m Normal Pulling pod/kube-auth-proxy-b649d648f-h99f8 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 60m Normal AddedInterface pod/kube-auth-proxy-b649d648f-s7zqw Add eth0 [10.133.0.26/23] from ovn-kubernetes 60m Normal Pulling pod/kube-auth-proxy-b649d648f-s7zqw Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 60m Normal SuccessfulCreate replicaset/kube-auth-proxy-b649d648f Created pod: kube-auth-proxy-b649d648f-h99f8 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-4ldkz 60m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-4ldkz Add eth0 [10.134.0.17/23] from ovn-kubernetes 60m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-4ldkz Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Normal Created pod/kube-auth-proxy-b649d648f-s7zqw Container created 60m Normal Pulled pod/kube-auth-proxy-b649d648f-h99f8 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.398s (2.398s including waiting). Image size: 188904677 bytes. 60m Normal Created pod/kube-auth-proxy-b649d648f-h99f8 Container created 60m Normal Started pod/kube-auth-proxy-b649d648f-h99f8 Container started 60m Normal Pulled pod/kube-auth-proxy-b649d648f-s7zqw Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.218s (2.218s including waiting). Image size: 188904677 bytes. 60m Normal Started pod/kube-auth-proxy-b649d648f-s7zqw Container started 60m Normal Started pod/istiod-openshift-gateway-7876bfcb74-4ldkz Container started 60m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-4ldkz Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.536s (2.536s including waiting). Image size: 185476477 bytes. 60m Normal Created pod/istiod-openshift-gateway-7876bfcb74-4ldkz Container created 60m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-67df64d57 from 0 to 1 60m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-67df64d57 Created pod: data-science-gateway-data-science-gateway-class-67df64d57-pzncp 60m Normal Pulling pod/data-science-gateway-data-science-gateway-class-67df64d57-pzncp Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 60m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-67df64d57-pzncp Add eth0 [10.132.0.36/23] from ovn-kubernetes 60m Normal Pulled pod/data-science-gateway-data-science-gateway-class-67df64d57-pzncp Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.434s (2.434s including waiting). Image size: 183555333 bytes. 60m Normal Created pod/data-science-gateway-data-science-gateway-class-67df64d57-pzncp Container created 60m Normal Started pod/data-science-gateway-data-science-gateway-class-67df64d57-pzncp Container started 60m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-kg9ds Add eth0 [10.134.0.19/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 59m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-kg9ds 59m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-kg9ds Container started 59m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-kg9ds Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 59m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-kg9ds Container created 59m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-4ldkz Stopping container discovery 59m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 59m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-4ldkz 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 0 to 1 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Add eth0 [10.132.0.42/23] from ovn-kubernetes 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Readiness probe failed: Get "http://10.132.0.42:15021/healthz/ready": read tcp 10.132.0.2:54266->10.132.0.42:15021: read: connection reset by peer 55m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Readiness probe error: Get "http://10.132.0.42:15021/healthz/ready": read tcp 10.132.0.2:54266->10.132.0.42:15021: read: connection reset by peer... 55m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Container created 55m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Container started 55m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Startup probe error: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Startup probe failed: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-zt6f2 54m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-8q5x4 54m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 54m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 54m Normal AddedInterface pod/payload-processing-5f78687d5b-zt6f2 Add eth0 [10.134.0.39/23] from ovn-kubernetes 54m Normal Pulling pod/payload-pre-processing-7b98bb678c-8q5x4 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 54m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-8q5x4 Add eth0 [10.133.0.43/23] from ovn-kubernetes 54m Normal Pulling pod/payload-processing-5f78687d5b-zt6f2 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 54m Normal Pulling pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal Pulling pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j 54m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 1 to 4 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn 54m Normal Pulled pod/payload-processing-5f78687d5b-zt6f2 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.8s (1.8s including waiting). Image size: 125317101 bytes. 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Add eth0 [10.133.0.44/23] from ovn-kubernetes 54m Normal Created pod/payload-processing-5f78687d5b-zt6f2 Container created 54m Normal Started pod/payload-processing-5f78687d5b-zt6f2 Container started 54m Normal Pulled pod/payload-pre-processing-7b98bb678c-8q5x4 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.829s (1.829s including waiting). Image size: 125317101 bytes. 54m Normal Started pod/payload-pre-processing-7b98bb678c-8q5x4 Container started 54m Normal Pulling pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Add eth0 [10.134.0.40/23] from ovn-kubernetes 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Add eth0 [10.134.0.41/23] from ovn-kubernetes 54m Normal Created pod/payload-pre-processing-7b98bb678c-8q5x4 Container created 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Container created 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Container created 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.83s (2.83s including waiting). Image size: 183555333 bytes. 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.451s (2.451s including waiting). Image size: 183555333 bytes. 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Container created 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Container started 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.827s (2.827s including waiting). Image size: 183555333 bytes. 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Container started 54m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 54m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Container started 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Container created 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Container started 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Container created 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Container created 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Container created 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Container started 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Add eth0 [10.133.0.47/23] from ovn-kubernetes 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Add eth0 [10.132.0.46/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-45vtp 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Add eth0 [10.132.0.45/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 4 to 8 54m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Add eth0 [10.133.0.46/23] from ovn-kubernetes 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Container started 54m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused... 54m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Container created 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Add eth0 [10.134.0.43/23] from ovn-kubernetes 54m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Container created 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 8 to 10 54m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Add eth0 [10.134.0.42/23] from ovn-kubernetes 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr 54m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 (combined from similar events): Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 53m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh 53m Normal AddedLabel gateway/e2e-worker-w4-4a0287 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-4a0287 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled up replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 0 to 1 53m Normal AddedLabel gateway/e2e-worker-w5-0d12c5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-0d12c5 53m Normal AddedLabel gateway/e2e-worker-w1-879596 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-879596 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled up replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 0 to 1 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled up replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 0 to 1 53m Normal AddedLabel gateway/e2e-worker-w2-8d3632 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-8d3632 53m Normal AddedLabel gateway/e2e-worker-w0-40cf9a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-40cf9a 53m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Add eth0 [10.133.0.48/23] from ovn-kubernetes 53m Normal AddedLabel gateway/e2e-worker-w6-92fd86 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-92fd86 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled up replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 0 to 1 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled up replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 0 to 1 53m Normal AddedLabel gateway/e2e-worker-w3-95672c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-95672c 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 53m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Add eth0 [10.133.0.51/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Add eth0 [10.133.0.49/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk 53m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Add eth0 [10.134.0.45/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled up replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 0 to 1 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr 53m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Add eth0 [10.133.0.50/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled up replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 0 to 1 53m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Add eth0 [10.134.0.46/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Add eth0 [10.134.0.44/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-4a0287 Scaled up replica set payload-processing-e2e-worker-w4-4a0287-cc6d67c5d from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d Error creating: pods "payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-4a0287: serviceaccount "payload-processing-e2e-worker-w4-4a0287" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-4a0287 Scaled up replica set payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f Created pod: payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f Error creating: pods "payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-4a0287: serviceaccount "payload-processing-e2e-worker-w4-4a0287" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d Created pod: payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 53m Normal AddedInterface pod/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 Add eth0 [10.133.0.52/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw Container created 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw Add eth0 [10.134.0.47/23] from ovn-kubernetes 53m Normal Started pod/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw Container started 53m Normal Started pod/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 Container started 53m Normal Pulled pod/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 Container created 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f Created pod: payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq 53m Normal AddedInterface pod/payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s Add eth0 [10.133.0.53/23] from ovn-kubernetes 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-8d3632-c779784c8 Error creating: pods "payload-processing-e2e-worker-w2-8d3632-c779784c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-8d3632: serviceaccount "payload-processing-e2e-worker-w2-8d3632" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-8d3632-c779784c8 Created pod: payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f Error creating: pods "payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-8d3632: serviceaccount "payload-processing-e2e-worker-w2-8d3632" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-8d3632 Scaled up replica set payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-8d3632 Scaled up replica set payload-processing-e2e-worker-w2-8d3632-c779784c8 from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b Error creating: pods "payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-40cf9a: serviceaccount "payload-processing-e2e-worker-w0-40cf9a" not found 53m Normal Created pod/payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s Container created 53m Normal AddedInterface pod/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 Add eth0 [10.134.0.50/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp Add eth0 [10.133.0.54/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp Container created 53m Normal Started pod/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp Container started 53m Normal Pulled pod/payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff Error creating: pods "payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-40cf9a: serviceaccount "payload-processing-e2e-worker-w0-40cf9a" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff Created pod: payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-40cf9a Scaled up replica set payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff from 0 to 1 53m Normal Created pod/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 Container created 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-40cf9a Scaled up replica set payload-processing-e2e-worker-w0-40cf9a-579fb97c7b from 0 to 1 53m Normal Started pod/payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s Container started 53m Normal Started pod/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq Container started 53m Normal Created pod/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq Container created 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 Container started 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b Created pod: payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq Add eth0 [10.134.0.49/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-0d12c5 Scaled up replica set payload-processing-e2e-worker-w5-0d12c5-dc776f966 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-0d12c5-dc776f966 Created pod: payload-processing-e2e-worker-w5-0d12c5-dc776f966-t282d 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-0d12c5-dc776f966 Error creating: pods "payload-processing-e2e-worker-w5-0d12c5-dc776f966-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-0d12c5: serviceaccount "payload-processing-e2e-worker-w5-0d12c5" not found 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775 Error creating: pods "payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-0d12c5: serviceaccount "payload-processing-e2e-worker-w5-0d12c5" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-0d12c5 Scaled up replica set payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775 Created pod: payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-mvczb 53m Normal Pulled pod/payload-processing-e2e-worker-w5-0d12c5-dc776f966-t282d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-mvczb Container started 53m Normal AddedInterface pod/payload-processing-e2e-worker-w5-0d12c5-dc776f966-t282d Add eth0 [10.133.0.55/23] from ovn-kubernetes 53m Normal Created pod/payload-processing-e2e-worker-w5-0d12c5-dc776f966-t282d Container created 53m Normal Started pod/payload-processing-e2e-worker-w5-0d12c5-dc776f966-t282d Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-mvczb Add eth0 [10.133.0.56/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-mvczb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-mvczb Container created 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-95672c-7dcb6ff684 Error creating: pods "payload-processing-e2e-worker-w3-95672c-7dcb6ff684-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-95672c: serviceaccount "payload-processing-e2e-worker-w3-95672c" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb Created pod: payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-92fd86 Scaled up replica set payload-processing-e2e-worker-w6-92fd86-6b4699fcfb from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-92fd86 Scaled up replica set payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999 from 0 to 1 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz Add eth0 [10.133.0.57/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999 Created pod: payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999 Error creating: pods "payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-92fd86: serviceaccount "payload-processing-e2e-worker-w6-92fd86" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-95672c Scaled up replica set payload-processing-e2e-worker-w3-95672c-7dcb6ff684 from 0 to 1 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-95672c Scaled up replica set payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75 from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb Error creating: pods "payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-92fd86: serviceaccount "payload-processing-e2e-worker-w6-92fd86" not found 53m Normal AddedInterface pod/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 Add eth0 [10.133.0.58/23] from ovn-kubernetes 53m Normal Created pod/payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb Container created 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-879596 Scaled up replica set payload-pre-processing-e2e-worker-w1-879596-5b795898bc from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-879596 Scaled up replica set payload-processing-e2e-worker-w1-879596-7f976c6496 from 0 to 1 53m Normal Created pod/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx Container created 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75 Created pod: payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx 53m Normal Pulled pod/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-95672c-7dcb6ff684 Created pod: payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb 53m Normal Created pod/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz Container created 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75 Error creating: pods "payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-95672c: serviceaccount "payload-processing-e2e-worker-w3-95672c" not found 53m Normal Started pod/payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb Container started 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-879596-7f976c6496 Error creating: pods "payload-processing-e2e-worker-w1-879596-7f976c6496-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-879596: serviceaccount "payload-processing-e2e-worker-w1-879596" not found 53m Normal Pulled pod/payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz Container started 53m Normal AddedInterface pod/payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb Add eth0 [10.133.0.59/23] from ovn-kubernetes 53m Normal Created pod/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 Container created 53m Normal Started pod/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx Add eth0 [10.133.0.60/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-879596-5b795898bc Created pod: payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-879596-5b795898bc Error creating: pods "payload-pre-processing-e2e-worker-w1-879596-5b795898bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-879596: serviceaccount "payload-processing-e2e-worker-w1-879596" not found 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf Container created 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-879596-7f976c6496 Created pod: payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf 53m Normal Created pod/payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj Container created 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj Add eth0 [10.133.0.61/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf Add eth0 [10.133.0.62/23] from ovn-kubernetes 53m Normal Started pod/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx Container started 53m Normal Pulled pod/payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf Container started 53m Normal Started pod/payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj Container started 53m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Container started 53m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Readiness probe failed: Get "http://10.133.0.48:15021/healthz/ready": read tcp 10.133.0.2:33662->10.133.0.48:15021: read: connection reset by peer 53m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Readiness probe error: Get "http://10.133.0.48:15021/healthz/ready": read tcp 10.133.0.2:33662->10.133.0.48:15021: read: connection reset by peer... 53m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Container created 53m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused 53m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Container created 53m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Container started 53m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Container created 53m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Container started 53m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Container created 53m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Container started 53m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Container started 53m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Container created 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled up replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 1 to 3 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Add eth0 [10.132.0.49/23] from ovn-kubernetes 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Container created 53m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Container started 53m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Add eth0 [10.134.0.55/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Container created 53m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused 53m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Container started 53m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Readiness probe error: Get "http://10.133.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Readiness probe failed: Get "http://10.133.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused... 53m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Container started 53m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Container created 53m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused... 53m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Container started 53m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 53m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Container created 52m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled up replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 1 to 2 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Add eth0 [10.133.0.67/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Container started 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled up replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 1 to 2 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Container created 52m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Container created 52m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Add eth0 [10.132.0.53/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Container started 52m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled up replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 1 to 2 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk 52m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Container started 52m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Container created 52m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Add eth0 [10.134.0.56/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled up replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 1 to 2 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled up replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 1 to 2 51m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Add eth0 [10.134.0.58/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Add eth0 [10.133.0.69/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Container created 51m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Container started 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled up replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 2 to 3 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 51m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Container created 51m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Container started 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b 51m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Add eth0 [10.132.0.59/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Container started 51m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Container created 51m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled up replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 1 to 2 51m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Add eth0 [10.134.0.59/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Container created 51m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Container started 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn 51m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 51m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled up replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 3 to 4 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Container created 51m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Container started 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled up replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 2 to 4 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x 51m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 Container started 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll 51m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 Container created 51m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Add eth0 [10.134.0.60/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 Add eth0 [10.132.0.64/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Add eth0 [10.133.0.82/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Container created 51m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Container started 51m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled up replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 2 to 3 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl 50m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Container created 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled up replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 2 to 4 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl Container created 50m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl Container started 50m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Add eth0 [10.134.0.62/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl Add eth0 [10.132.0.66/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled up replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 2 to 3 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled up replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 2 to 4 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs 50m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Add eth0 [10.132.0.68/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw 50m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Container started 50m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Container started 50m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Container created 50m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Add eth0 [10.132.0.67/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg 50m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Container started 50m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Container created 50m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Container started 50m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Container created 50m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Add eth0 [10.133.0.90/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Add eth0 [10.133.0.89/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Container created 50m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Container started 50m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled up replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 3 to 4 50m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Add eth0 [10.133.0.91/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Container started 50m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Container created 50m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Startup probe failed: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Startup probe error: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused... 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 50m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Container started 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled up replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 4 to 6 50m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Add eth0 [10.133.0.98/23] from ovn-kubernetes 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 50m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Container created 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt 50m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Container created 50m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Container started 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled up replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 3 to 5 50m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Add eth0 [10.134.0.65/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled up replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 3 to 6 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 50m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Add eth0 [10.134.0.67/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Container started 50m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Container started 50m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Add eth0 [10.133.0.99/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Container started 50m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Container started 50m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Container created 50m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Add eth0 [10.132.0.72/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Container created 50m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Container started 50m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Add eth0 [10.134.0.66/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Add eth0 [10.132.0.71/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Container created 50m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Container created 50m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Container created 50m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled up replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 4 to 6 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 49m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 Add eth0 [10.132.0.74/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled up replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 4 to 7 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 Add eth0 [10.133.0.102/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 Container created 49m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Container started 49m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Container started 49m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Container created 49m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Add eth0 [10.134.0.68/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Container started 49m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Container created 49m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Container created 49m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Add eth0 [10.134.0.69/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 Container created 49m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 Container started 49m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 Container started 49m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Add eth0 [10.132.0.75/23] from ovn-kubernetes 49m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 49m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Add eth0 [10.134.0.71/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 49m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Container created 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr 49m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr Add eth0 [10.132.0.76/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled up replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 4 to 7 49m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Add eth0 [10.133.0.104/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 49m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled up replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 6 to 7 49m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Container started 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 49m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr Container created 49m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt 49m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr Container started 49m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Container started 49m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Add eth0 [10.134.0.72/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Container created 49m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Container started 49m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Container created 49m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Startup probe error: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Startup probe failed: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled up replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 7 to 10 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled up replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 4 to 8 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 (combined from similar events): Created pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Add eth0 [10.132.0.77/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Container created 49m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Container created 49m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Add eth0 [10.133.0.107/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Container started 49m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Container created 49m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Container started 49m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Container started 49m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Container created 49m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Container started 49m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Add eth0 [10.133.0.108/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Container started 49m Normal Created pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Container created 49m Normal Pulled pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Add eth0 [10.133.0.109/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Add eth0 [10.132.0.78/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Container started 49m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Add eth0 [10.134.0.74/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Container created 49m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Container started 49m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Container created 49m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Add eth0 [10.134.0.73/23] from ovn-kubernetes 49m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Startup probe error: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Startup probe failed: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Startup probe error: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Startup probe failed: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 (combined from similar events): Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 (combined from similar events): Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Created pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 49m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf Add eth0 [10.132.0.80/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled up replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 5 to 10 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled up replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 8 to 10 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Created pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx 49m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl Container created 49m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Container created 49m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Container started 49m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt Container created 49m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Container started 49m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx Container created 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc (combined from similar events): Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c 49m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Add eth0 [10.132.0.81/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx Container started 49m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j Add eth0 [10.133.0.113/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Add eth0 [10.134.0.78/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Container started 49m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Container created 49m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl Add eth0 [10.133.0.115/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl Container started 49m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx Add eth0 [10.134.0.75/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl 49m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j Container created 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal Pulled pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Created pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt 49m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j Container started 49m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Add eth0 [10.132.0.82/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled up replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 6 to 10 49m Normal AddedInterface pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Add eth0 [10.134.0.76/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Container started 49m Normal Created pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Container created 49m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt Add eth0 [10.134.0.77/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Add eth0 [10.133.0.114/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Container created 49m Normal Pulled pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf Container started 49m Normal Created pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf Container created 49m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Add eth0 [10.132.0.83/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Container started 49m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt Container started 49m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Container created 49m Normal Started pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Container started 49m Normal Created pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Container created 49m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j Startup probe error: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused... 49m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-xc4fv 49m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j Startup probe failed: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused 49m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 49m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 49m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused 49m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Add eth0 [10.133.0.118/23] from ovn-kubernetes 49m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Startup probe failed: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Startup probe error: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 48m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Container created 48m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 48m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.584s (3.584s including waiting). Image size: 185476477 bytes. 48m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 48m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Container started 48m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 48m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 48m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 (combined from similar events): Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Created pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled up replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 7 to 10 48m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Add eth0 [10.134.0.79/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Container started 48m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 (combined from similar events): Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Container created 48m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Container created 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc 48m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Container started 48m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Container started 48m Normal Started pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Container started 48m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Add eth0 [10.132.0.86/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Created pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 48m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Add eth0 [10.134.0.80/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Container created 48m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Add eth0 [10.133.0.125/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled up replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 6 to 10 48m Normal AddedInterface pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Add eth0 [10.133.0.123/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Container started 48m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Container created 48m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Add eth0 [10.132.0.87/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Container created 48m Normal Pulled pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Container started 48m Normal Started pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Container started 48m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Container created 48m Normal Created pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Container created 48m Normal AddedInterface pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Add eth0 [10.134.0.81/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Startup probe failed: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Startup probe error: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Startup probe failed: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Startup probe error: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Readiness probe failed: Get "http://10.134.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Readiness probe error: Get "http://10.134.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Readiness probe error: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Readiness probe failed: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Readiness probe error: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Readiness probe failed: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 (combined from similar events): Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled up replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 7 to 10 48m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Add eth0 [10.132.0.90/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm Add eth0 [10.134.0.82/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h Add eth0 [10.133.0.130/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm Container started 48m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm Container created 48m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h Container started 48m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h Container created 48m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Container created 48m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Container started 48m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h Startup probe failed: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h Startup probe error: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Readiness probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Readiness probe error: Get "http://10.133.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Readiness probe failed: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Readiness probe error: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Readiness probe failed: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Readiness probe failed: Get "http://10.133.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Readiness probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Readiness probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Readiness probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Readiness probe error: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Readiness probe failed: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Readiness probe error: Get "http://10.133.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Readiness probe failed: Get "http://10.133.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Readiness probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Readiness probe failed: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Readiness probe error: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Readiness probe failed: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Readiness probe error: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Readiness probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Readiness probe error: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Readiness probe failed: Get "http://10.133.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Readiness probe error: Get "http://10.133.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Readiness probe failed: Get "http://10.133.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Readiness probe error: Get "http://10.133.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Readiness probe failed: Get "http://10.133.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Readiness probe error: Get "http://10.133.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Readiness probe failed: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Readiness probe error: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Readiness probe failed: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Readiness probe error: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal ScalingReplicaSet deployment/e2e-ait-1309d608-openshift-default Scaled up replica set e2e-ait-1309d608-openshift-default-cfc7dfc5 from 0 to 1 47m Normal AddedLabel gateway/e2e-ait-1309d608 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1309d608 47m Normal SuccessfulCreate replicaset/e2e-ait-1309d608-openshift-default-cfc7dfc5 Created pod: e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv 47m Normal Pulled pod/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv Add eth0 [10.134.0.83/23] from ovn-kubernetes 47m Normal Created pod/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv Container created 47m Normal Started pod/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv Container started 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-1309d608-65d9d8b479 Error creating: pods "payload-pre-processing-e2e-ait-1309d608-65d9d8b479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1309d608: serviceaccount "payload-processing-e2e-ait-1309d608" not found 47m Warning FailedCreate replicaset/payload-processing-e2e-ait-1309d608-6db75d7cd4 Error creating: pods "payload-processing-e2e-ait-1309d608-6db75d7cd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1309d608: serviceaccount "payload-processing-e2e-ait-1309d608" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-1309d608 Scaled up replica set payload-pre-processing-e2e-ait-1309d608-65d9d8b479 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-1309d608 Scaled up replica set payload-processing-e2e-ait-1309d608-6db75d7cd4 from 0 to 1 47m Normal AddedInterface pod/payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c Add eth0 [10.134.0.84/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-1309d608-65d9d8b479 Created pod: payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-1309d608-6db75d7cd4 Created pod: payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph 47m Normal Started pod/payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c Container started 47m Normal AddedLabel gateway/e2e-migrate-3593b8ba-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-3593b8ba-gw 47m Normal AddedInterface pod/payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph Add eth0 [10.134.0.85/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph Container created 47m Normal Started pod/payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph Container started 47m Normal Pulled pod/payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c Container created 47m Normal ScalingReplicaSet deployment/e2e-migrate-3593b8ba-gw-openshift-default Scaled up replica set e2e-migrate-3593b8ba-gw-openshift-default-594887f8df from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df Created pod: e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-btd2n 47m Normal AddedInterface pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-btd2n Add eth0 [10.134.0.86/23] from ovn-kubernetes 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f Error creating: pods "payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-3593b8ba: serviceaccount "payload-processing-e2e-migrate-3593b8ba" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-3593b8ba Scaled up replica set payload-processing-e2e-migrate-3593b8ba-66df9dddf4 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-3593b8ba-66df9dddf4 Created pod: payload-processing-e2e-migrate-3593b8ba-66df9dddf4-5rbrv 47m Warning FailedCreate replicaset/payload-processing-e2e-migrate-3593b8ba-66df9dddf4 Error creating: pods "payload-processing-e2e-migrate-3593b8ba-66df9dddf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-3593b8ba: serviceaccount "payload-processing-e2e-migrate-3593b8ba" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-3593b8ba Scaled up replica set payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f Created pod: payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-fr5fg 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1309d608-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1309d608-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Readiness probe failed: Get "http://10.134.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Readiness probe error: Get "http://10.134.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Readiness probe error: Get "http://10.132.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Readiness probe error: Get "http://10.134.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Readiness probe failed: Get "http://10.133.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Readiness probe error: Get "http://10.133.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Readiness probe failed: Get "http://10.133.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Readiness probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Readiness probe error: Get "http://10.133.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Readiness probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Readiness probe error: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Readiness probe failed: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Readiness probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Readiness probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Readiness probe failed: Get "http://10.134.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Readiness probe failed: Get "http://10.133.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Readiness probe error: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Readiness probe failed: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Readiness probe error: Get "http://10.133.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Readiness probe error: Get "http://10.133.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Readiness probe error: Get "http://10.133.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Readiness probe failed: Get "http://10.133.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Readiness probe failed: Get "http://10.133.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Readiness probe failed: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Readiness probe error: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Readiness probe failed: Get "http://10.132.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Readiness probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Readiness probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Readiness probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Readiness probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal Pulled pod/payload-processing-e2e-migrate-3593b8ba-66df9dddf4-5rbrv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-migrate-3593b8ba-66df9dddf4-5rbrv Add eth0 [10.134.0.87/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-fr5fg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-fr5fg Container started 47m Normal Started pod/payload-processing-e2e-migrate-3593b8ba-66df9dddf4-5rbrv Container started 47m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-fr5fg Add eth0 [10.134.0.88/23] from ovn-kubernetes 47m Normal Created pod/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-fr5fg Container created 47m Normal Created pod/payload-processing-e2e-migrate-3593b8ba-66df9dddf4-5rbrv Container created 47m Normal SuccessfulCreate replicaset/e2e-mt-f922c6e9-openshift-default-797b4f7c65 Created pod: e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw 47m Normal AddedLabel gateway/e2e-mt-f922c6e9 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-f922c6e9 47m Normal ScalingReplicaSet deployment/e2e-mt-f922c6e9-openshift-default Scaled up replica set e2e-mt-f922c6e9-openshift-default-797b4f7c65 from 0 to 1 47m Normal AddedInterface pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Add eth0 [10.132.0.96/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-f922c6e9 Scaled up replica set payload-pre-processing-e2e-mt-f922c6e9-6854bb686c from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-f922c6e9 Scaled up replica set payload-processing-e2e-mt-f922c6e9-55f64bd4b8 from 0 to 1 47m Normal Pulled pod/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-n7lx5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedCreate replicaset/payload-processing-e2e-mt-f922c6e9-55f64bd4b8 Error creating: pods "payload-processing-e2e-mt-f922c6e9-55f64bd4b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-f922c6e9: serviceaccount "payload-processing-e2e-mt-f922c6e9" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c Created pod: payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-n7lx5 47m Normal Created pod/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-n7lx5 Container created 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c Error creating: pods "payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-f922c6e9: serviceaccount "payload-processing-e2e-mt-f922c6e9" not found 47m Normal AddedInterface pod/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-n7lx5 Add eth0 [10.133.0.143/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-n7lx5 Container started 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-f922c6e9-55f64bd4b8 Created pod: payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s 47m Normal AddedInterface pod/payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s Add eth0 [10.133.0.144/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s Container started 47m Normal Created pod/payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s Container created 47m Normal Pulled pod/payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1309d608-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1309d608-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal AddedLabel gateway/e2e-derive-591986bb Added label istio.io/rev=openshift-gateway to gateway e2e-derive-591986bb 47m Normal SuccessfulCreate replicaset/e2e-derive-591986bb-openshift-default-cb894bd5 Created pod: e2e-derive-591986bb-openshift-default-cb894bd5-5thqz 47m Normal ScalingReplicaSet deployment/e2e-derive-591986bb-openshift-default Scaled up replica set e2e-derive-591986bb-openshift-default-cb894bd5 from 0 to 1 46m Normal Started pod/e2e-derive-591986bb-openshift-default-cb894bd5-5thqz Container started 46m Normal Created pod/e2e-derive-591986bb-openshift-default-cb894bd5-5thqz Container created 46m Normal AddedInterface pod/e2e-derive-591986bb-openshift-default-cb894bd5-5thqz Add eth0 [10.133.0.145/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-derive-591986bb-openshift-default-cb894bd5-5thqz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning FailedCreate replicaset/payload-processing-e2e-derive-591986bb-8559c645b8 Error creating: pods "payload-processing-e2e-derive-591986bb-8559c645b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-591986bb: serviceaccount "payload-processing-e2e-derive-591986bb" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f Error creating: pods "payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-591986bb: serviceaccount "payload-processing-e2e-derive-591986bb" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f Created pod: payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-tx58r 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-591986bb Scaled up replica set payload-pre-processing-e2e-derive-591986bb-b7fb6c67f from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-591986bb-8559c645b8 Created pod: payload-processing-e2e-derive-591986bb-8559c645b8-92szf 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-591986bb Scaled up replica set payload-processing-e2e-derive-591986bb-8559c645b8 from 0 to 1 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-3593b8ba-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-3593b8ba-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Readiness probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Readiness probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal AddedInterface pod/payload-processing-e2e-derive-591986bb-8559c645b8-92szf Add eth0 [10.133.0.147/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-tx58r Add eth0 [10.133.0.146/23] from ovn-kubernetes 46m Warning Unhealthy pod/payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s Readiness probe failed: timeout: health rpc did not complete within 1s 46m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Readiness probe error: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal Pulled pod/payload-processing-e2e-derive-591986bb-8559c645b8-92szf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Readiness probe failed: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal Pulled pod/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-tx58r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-derive-591986bb-8559c645b8-92szf Container started 46m Normal Created pod/payload-processing-e2e-derive-591986bb-8559c645b8-92szf Container created 46m Normal Started pod/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-tx58r Container started 46m Normal Created pod/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-tx58r Container created 46m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-st7pn 46m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Add eth0 [10.132.0.99/23] from ovn-kubernetes 46m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-591986bb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-591986bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.919s (2.919s including waiting). Image size: 185476477 bytes. 46m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Container created 46m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Container started 46m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-f922c6e9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-f922c6e9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-591986bb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-591986bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Killing pod/payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph Stopping container payload-processing 46m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Readiness probe error: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Readiness probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Readiness probe failed: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Readiness probe failed: Get "http://10.133.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Readiness probe error: Get "http://10.133.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/payload-pre-processing-e2e-ait-1309d608-65d9d8b479-xp28c Readiness probe failed: timeout: failed to connect service "10.134.0.84:50052" within 1s: context deadline exceeded 46m Warning Unhealthy pod/payload-processing-e2e-ait-1309d608-6db75d7cd4-ctnph Readiness probe failed: timeout: failed to connect service "10.134.0.85:50052" within 1s: context deadline exceeded 46m Normal SuccessfulCreate replicaset/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df Created pod: e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q 46m Normal SuccessfulCreate replicaset/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df Created pod: e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 46m Normal ScalingReplicaSet deployment/e2e-migrate-3593b8ba-gw-openshift-default Scaled up replica set e2e-migrate-3593b8ba-gw-openshift-default-594887f8df from 1 to 3 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-migrate-3593b8ba-gw-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 46m Normal Created pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q Container created 46m Normal Started pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 Container started 46m Normal Created pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 Container created 46m Normal Killing pod/payload-processing-e2e-migrate-3593b8ba-66df9dddf4-5rbrv Stopping container payload-processing 46m Normal Pulled pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q Add eth0 [10.132.0.100/23] from ovn-kubernetes 46m Normal Killing pod/payload-pre-processing-e2e-migrate-3593b8ba-b5c889f7f-fr5fg Stopping container payload-pre-processing 46m Normal Pulled pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q Container started 46m Normal AddedInterface pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 Add eth0 [10.133.0.151/23] from ovn-kubernetes 46m Warning ProbeError pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal Killing pod/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-ait-e1d9591d-openshift-default Scaled up replica set e2e-ait-e1d9591d-openshift-default-7c64b644c7 from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-ait-e1d9591d-openshift-default-7c64b644c7 Created pod: e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw 46m Normal AddedLabel gateway/e2e-ait-e1d9591d Added label istio.io/rev=openshift-gateway to gateway e2e-ait-e1d9591d 46m Normal Created pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Container created 46m Normal Pulled pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Container started 46m Normal AddedInterface pod/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw Add eth0 [10.134.0.91/23] from ovn-kubernetes 46m Warning ProbeError pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 Startup probe error: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 Startup probe failed: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused 46m Warning ProbeError pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d Error creating: pods "payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e1d9591d: serviceaccount "payload-processing-e2e-ait-e1d9591d" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff Created pod: payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-pssgt 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff Error creating: pods "payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e1d9591d: serviceaccount "payload-processing-e2e-ait-e1d9591d" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-e1d9591d Scaled up replica set payload-processing-e2e-ait-e1d9591d-7c6c8cb89d from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-e1d9591d Scaled up replica set payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d Created pod: payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-fb88b 45m Warning ProbeError pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 45m Normal Created pod/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-pssgt Container created 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-pssgt Add eth0 [10.134.0.92/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-fb88b Add eth0 [10.133.0.152/23] from ovn-kubernetes 45m Warning ProbeError pod/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv Readiness probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal Pulled pod/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-pssgt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-fb88b Container started 45m Normal Created pod/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-fb88b Container created 45m Warning Unhealthy pod/e2e-ait-1309d608-openshift-default-cfc7dfc5-4nnqv Readiness probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal Started pod/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-pssgt Container started 45m Normal Pulled pod/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-fb88b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning Unhealthy pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-3593b8ba-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-3593b8ba-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Pulled pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-btd2n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-btd2n Container created 45m Normal Started pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-btd2n Container started 45m Normal Killing pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-msvx9 Stopping container istio-proxy 45m Normal Killing pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-btd2n Stopping container istio-proxy 45m Normal Killing pod/e2e-migrate-3593b8ba-gw-openshift-default-594887f8df-2829q Stopping container istio-proxy 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e1d9591d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e1d9591d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/payload-pre-processing-e2e-derive-591986bb-b7fb6c67f-tx58r Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-derive-591986bb-8559c645b8-92szf Stopping container payload-processing 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-f922c6e9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-f922c6e9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/e2e-derive-591986bb-openshift-default-cb894bd5-5thqz Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-36b46d-openshift-default Scaled up replica set e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 from 0 to 1 45m Normal AddedLabel gateway/e2e-shared-a-w3-36b46d Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-36b46d 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 Created pod: e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 45m Normal AddedInterface pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Add eth0 [10.134.0.94/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-36b46d Scaled up replica set payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c from 0 to 1 45m Normal Created pod/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj Container created 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b Error creating: pods "payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-36b46d: serviceaccount "payload-processing-e2e-shared-a-w3-36b46d" not found 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj Add eth0 [10.133.0.154/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c Created pod: payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 Add eth0 [10.133.0.155/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj Container started 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c Error creating: pods "payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-36b46d: serviceaccount "payload-processing-e2e-shared-a-w3-36b46d" not found 45m Normal Pulled pod/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-36b46d Scaled up replica set payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b Created pod: payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 Container started 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 Container created 45m Normal Killing pod/payload-pre-processing-e2e-ait-e1d9591d-58975cb8ff-pssgt Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-ait-e1d9591d-7c6c8cb89d-fb88b Stopping container payload-processing 45m Normal AddedLabel gateway/e2e-shared-a-w2-afdbe8 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-afdbe8 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-afdbe8-openshift-default Scaled up replica set e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c Created pod: e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll 45m Normal AddedInterface pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Add eth0 [10.133.0.158/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj Add eth0 [10.133.0.159/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-afdbe8 Scaled up replica set payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-afdbe8 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5 Created pod: payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4 Error creating: pods "payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-afdbe8: serviceaccount "payload-processing-e2e-shared-a-w2-afdbe8" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4 Created pod: payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-afdbe8: serviceaccount "payload-processing-e2e-shared-a-w2-afdbe8" not found 44m Normal Started pod/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl Container started 44m Normal Created pod/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj Container started 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl Add eth0 [10.134.0.96/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj Container created 44m Normal Pulled pod/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw Container started 44m Normal Created pod/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw Container created 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e1d9591d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e1d9591d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedLabel gateway/e2e-shared-b-w3-8f1627 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-8f1627 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f Created pod: e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-8f1627-openshift-default Scaled up replica set e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f from 0 to 1 44m Normal AddedInterface pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Add eth0 [10.134.0.97/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-8f1627 Scaled up replica set payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-8f1627: serviceaccount "payload-processing-e2e-shared-b-w3-8f1627" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6 Created pod: payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-8f1627 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975 Created pod: payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6 Error creating: pods "payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-8f1627: serviceaccount "payload-processing-e2e-shared-b-w3-8f1627" not found 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz Container created 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 Add eth0 [10.133.0.160/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz Container started 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz Add eth0 [10.133.0.161/23] from ovn-kubernetes 44m Normal Killing pod/e2e-ait-e1d9591d-openshift-default-7c64b644c7-kstxw Stopping container istio-proxy 44m Normal AddedInterface pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Add eth0 [10.133.0.163/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-e81b56-openshift-default Scaled up replica set e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69 Created pod: e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv 44m Normal AddedLabel gateway/e2e-shared-a-w6-e81b56 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-e81b56 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-afdbe8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-afdbe8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning ProbeError pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Readiness probe error: Get "http://10.134.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-e81b56 Scaled up replica set payload-processing-e2e-shared-a-w6-e81b56-598655d846 from 0 to 1 44m Warning Unhealthy pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Readiness probe failed: Get "http://10.134.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-e81b56-598655d846 Error creating: pods "payload-processing-e2e-shared-a-w6-e81b56-598655d846-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-e81b56: serviceaccount "payload-processing-e2e-shared-a-w6-e81b56" not found 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-e81b56: serviceaccount "payload-processing-e2e-shared-a-w6-e81b56" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9 Created pod: payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-e81b56 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-e81b56-598655d846 Created pod: payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m 44m Normal Created pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Container created 44m Normal Pulled pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m Container created 44m Normal Pulled pod/payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Container started 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m Add eth0 [10.134.0.98/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 Add eth0 [10.134.0.99/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 Container started 44m Normal Killing pod/payload-pre-processing-e2e-mt-f922c6e9-6854bb686c-n7lx5 Stopping container payload-pre-processing 44m Normal Killing pod/payload-processing-e2e-mt-f922c6e9-55f64bd4b8-kkk9s Stopping container payload-processing 44m Warning Unhealthy pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 44m Warning ProbeError pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-36b46d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-36b46d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Stopping container istio-proxy 44m Warning Unhealthy pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-mt-f922c6e9-openshift-default-797b4f7c65-dxvhw Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-84304f-openshift-default Scaled up replica set e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 Created pod: e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf 44m Normal AddedLabel gateway/e2e-shared-b-w2-84304f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-84304f 44m Normal AddedInterface pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Add eth0 [10.133.0.164/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x 44m Normal AddedLabel gateway/e2e-mt-d478b36e Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d478b36e 44m Normal ScalingReplicaSet deployment/e2e-mt-d478b36e-openshift-default Scaled up replica set e2e-mt-d478b36e-openshift-default-56b9ccb7bf from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f Created pod: e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x 44m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Add eth0 [10.133.0.165/23] from ovn-kubernetes 44m Normal AddedLabel gateway/e2e-shared-a-w0-c7b96a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-c7b96a 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-c7b96a-openshift-default Scaled up replica set e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f from 0 to 1 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-36b46d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-36b46d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedInterface pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Add eth0 [10.132.0.102/23] from ovn-kubernetes 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b Error creating: pods "payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-84304f: serviceaccount "payload-processing-e2e-shared-b-w2-84304f" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-84304f Scaled up replica set payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b Created pod: payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-84304f Scaled up replica set payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff Error creating: pods "payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-84304f: serviceaccount "payload-processing-e2e-shared-b-w2-84304f" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff Created pod: payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb 43m Normal Pulled pod/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z Container created 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-d478b36e Scaled up replica set payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5 from 0 to 1 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z Add eth0 [10.134.0.102/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb Add eth0 [10.133.0.166/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb Container created 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb Container started 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-d478b36e Scaled up replica set payload-processing-e2e-mt-d478b36e-65b9f9c44b from 0 to 1 43m Normal AddedInterface pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg Add eth0 [10.133.0.167/23] from ovn-kubernetes 43m Normal Created pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg Container created 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5 Created pod: payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg 43m Warning Unhealthy pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Readiness probe failed: Get "http://10.133.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-d478b36e-65b9f9c44b Created pod: payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n 43m Warning ProbeError pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Readiness probe error: Get "http://10.133.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning FailedCreate replicaset/payload-processing-e2e-mt-d478b36e-65b9f9c44b Error creating: pods "payload-processing-e2e-mt-d478b36e-65b9f9c44b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d478b36e: serviceaccount "payload-processing-e2e-mt-d478b36e" not found 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5 Error creating: pods "payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d478b36e: serviceaccount "payload-processing-e2e-mt-d478b36e" not found 43m Normal Pulled pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg Container started 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976 Error creating: pods "payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-c7b96a: serviceaccount "payload-processing-e2e-shared-a-w0-c7b96a" not found 43m Normal Started pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n Container started 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-c7b96a Scaled up replica set payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976 from 0 to 1 43m Normal Pulled pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n Container created 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-c7b96a Scaled up replica set payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5 Created pod: payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-l7s7s 43m Normal AddedInterface pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n Add eth0 [10.134.0.103/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976 Created pod: payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-54nxc 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-c7b96a: serviceaccount "payload-processing-e2e-shared-a-w0-c7b96a" not found 43m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-l7s7s Container started 43m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-l7s7s Container created 43m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-l7s7s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-l7s7s Add eth0 [10.134.0.104/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-54nxc Add eth0 [10.134.0.105/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-54nxc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Container started 43m Normal Created pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Container created 43m Normal Pulled pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-54nxc Container created 43m Normal Started pod/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-54nxc Container started 43m Warning Unhealthy pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Startup probe failed: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Startup probe error: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused... 43m Normal SuccessfulCreate replicaset/e2e-api-a-0ab33e-openshift-default-65bc44c749 Created pod: e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h 43m Normal AddedLabel gateway/e2e-api-a-0ab33e Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-0ab33e 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-6bb06a-openshift-default Scaled up replica set e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 from 0 to 1 43m Normal AddedLabel gateway/e2e-shared-b-w6-6bb06a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-6bb06a 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Created pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn 43m Normal ScalingReplicaSet deployment/e2e-api-a-0ab33e-openshift-default Scaled up replica set e2e-api-a-0ab33e-openshift-default-65bc44c749 from 0 to 1 43m Normal AddedInterface pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Add eth0 [10.132.0.104/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Add eth0 [10.133.0.168/23] from ovn-kubernetes 43m Normal AddedLabel gateway/e2e-api-a-f76fb5 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-f76fb5 43m Normal ScalingReplicaSet deployment/e2e-api-a-f76fb5-openshift-default Scaled up replica set e2e-api-a-f76fb5-openshift-default-7755f59d9c from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-api-a-f76fb5-openshift-default-7755f59d9c Created pod: e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph 43m Normal AddedInterface pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Add eth0 [10.132.0.105/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459 Created pod: payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-frxg9 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459 Error creating: pods "payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0ab33e: serviceaccount "payload-processing-e2e-api-a-0ab33e" not found 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-0ab33e Scaled up replica set payload-processing-e2e-api-a-0ab33e-54f7dbfcc8 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8 Created pod: payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-bsvsw 43m Warning FailedCreate replicaset/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8 Error creating: pods "payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0ab33e: serviceaccount "payload-processing-e2e-api-a-0ab33e" not found 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-c7b96a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-c7b96a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-0ab33e Scaled up replica set payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459 from 0 to 1 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-frxg9 Add eth0 [10.133.0.169/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-frxg9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-frxg9 Container created 43m Normal Pulled pod/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-bsvsw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-bsvsw Container created 43m Normal Started pod/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-bsvsw Container started 43m Normal AddedInterface pod/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-bsvsw Add eth0 [10.134.0.107/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-frxg9 Container started 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-6bb06a Scaled up replica set payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb Error creating: pods "payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-6bb06a: serviceaccount "payload-processing-e2e-shared-b-w6-6bb06a" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-6bb06a Scaled up replica set payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf from 0 to 1 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-afdbe8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-afdbe8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb Created pod: payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf Error creating: pods "payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-6bb06a: serviceaccount "payload-processing-e2e-shared-b-w6-6bb06a" not found 43m Normal Pulled pod/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb Add eth0 [10.133.0.170/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf Created pod: payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs Add eth0 [10.134.0.108/23] from ovn-kubernetes 43m Normal Created pod/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb Container created 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs Container created 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs Container started 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-f76fb5 Scaled up replica set payload-processing-e2e-api-a-f76fb5-5d75b85b9d from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-f76fb5 Scaled up replica set payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9 Created pod: payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-6h728 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9 Error creating: pods "payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f76fb5: serviceaccount "payload-processing-e2e-api-a-f76fb5" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-api-a-f76fb5-5d75b85b9d Error creating: pods "payload-processing-e2e-api-a-f76fb5-5d75b85b9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f76fb5: serviceaccount "payload-processing-e2e-api-a-f76fb5" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-f76fb5-5d75b85b9d Created pod: payload-processing-e2e-api-a-f76fb5-5d75b85b9d-64hcd 43m Normal AddedInterface pod/payload-processing-e2e-api-a-f76fb5-5d75b85b9d-64hcd Add eth0 [10.133.0.171/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-6h728 Add eth0 [10.133.0.172/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-api-a-f76fb5-5d75b85b9d-64hcd Container started 43m Normal Created pod/payload-processing-e2e-api-a-f76fb5-5d75b85b9d-64hcd Container created 43m Normal Created pod/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-6h728 Container created 43m Normal Pulled pod/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-6h728 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-api-a-f76fb5-5d75b85b9d-64hcd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-6h728 Container started 43m Normal Created pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Container created 43m Normal Pulled pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Container started 43m Normal Started pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Container started 43m Normal Created pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Container created 43m Normal Pulled pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-e81b56-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-e81b56-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Startup probe failed: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Startup probe error: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-0ab33e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-0ab33e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/e2e-inf-a-a26426-openshift-default Scaled up replica set e2e-inf-a-a26426-openshift-default-6766c9bbf from 0 to 1 43m Normal AddedLabel gateway/e2e-inf-a-a26426 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-a26426 43m Normal SuccessfulCreate replicaset/e2e-inf-a-a26426-openshift-default-6766c9bbf Created pod: e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-8f1627-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-8f1627-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal AddedInterface pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Add eth0 [10.133.0.173/23] from ovn-kubernetes 43m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 43m Warning Unhealthy pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Readiness probe failed: Get "http://10.133.0.164:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Readiness probe error: Get "http://10.133.0.164:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-z6q2x 43m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-z6q2x Container started 43m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-z6q2x Container created 43m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-z6q2x Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-z6q2x Add eth0 [10.133.0.174/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Readiness probe failed: Get "http://10.132.0.102:15021/healthz/ready": read tcp 10.132.0.2:46996->10.132.0.102:15021: read: connection reset by peer 43m Warning ProbeError pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Readiness probe error: Get "http://10.132.0.102:15021/healthz/ready": read tcp 10.132.0.2:46996->10.132.0.102:15021: read: connection reset by peer... 43m Normal Pulled pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-z6q2x Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Readiness probe failed: Get "http://10.133.0.165:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal Started pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Container started 42m Normal Created pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Container created 42m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Readiness probe error: Get "http://10.133.0.165:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-z6q2x Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal Started pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Container started 42m Normal Pulled pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Container created 42m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Container started 42m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Container created 42m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-a26426 Scaled up replica set payload-processing-e2e-inf-a-a26426-6b9b8c6db6 from 0 to 1 42m Warning Unhealthy pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-a26426-6b9b8c6db6 Error creating: pods "payload-processing-e2e-inf-a-a26426-6b9b8c6db6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a26426: serviceaccount "payload-processing-e2e-inf-a-a26426" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-a26426-6b9b8c6db6 Created pod: payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7 Created pod: payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-a26426 Scaled up replica set payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7 Error creating: pods "payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a26426: serviceaccount "payload-processing-e2e-inf-a-a26426" not found 42m Warning ProbeError pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 42m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Deleted pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 42m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Deleted pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled down replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 10 to 5 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 5; reason: All metrics below target 42m Normal Pulled pod/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Deleted pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Deleted pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-jcvll 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Deleted pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-wzzjm 42m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4vtrh Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nfmw9 Stopping container istio-proxy 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 Add eth0 [10.133.0.175/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn Container started 42m Normal Pulled pod/payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 Container started 42m Normal Created pod/payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn Container created 42m Normal Created pod/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 Container created 42m Normal AddedInterface pod/payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn Add eth0 [10.133.0.176/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Startup probe error: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Startup probe failed: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-84304f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-84304f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Startup probe failed: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-e81b56-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-e81b56-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d478b36e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Startup probe error: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused... 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d478b36e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-2zp47 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-8f1627-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal AddedLabel gateway/e2e-shared-b-w0-e526ec Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-e526ec 42m Normal AddedInterface pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Add eth0 [10.132.0.111/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-e526ec-openshift-default Scaled up replica set e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df from 0 to 1 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-8f1627-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df Created pod: e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46 Created pod: payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-r84hx 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-e526ec Scaled up replica set payload-processing-e2e-shared-b-w0-e526ec-59659cfd46 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-e526ec: serviceaccount "payload-processing-e2e-shared-b-w0-e526ec" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9 Created pod: payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-g5fq9 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46 Error creating: pods "payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-e526ec: serviceaccount "payload-processing-e2e-shared-b-w0-e526ec" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-e526ec Scaled up replica set payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9 from 0 to 1 42m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-r84hx Add eth0 [10.134.0.109/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-r84hx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-r84hx Container created 42m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-g5fq9 Container created 42m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-g5fq9 Container started 42m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-g5fq9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-g5fq9 Add eth0 [10.134.0.110/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-r84hx Container started 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d478b36e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d478b36e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-c7b96a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-c7b96a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal SuccessfulCreate replicaset/e2e-api-a-0ab33e-openshift-default-65bc44c749 Created pod: e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-0ab33e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 42m Normal ScalingReplicaSet deployment/e2e-api-a-0ab33e-openshift-default Scaled up replica set e2e-api-a-0ab33e-openshift-default-65bc44c749 from 1 to 4 42m Normal SuccessfulCreate replicaset/e2e-api-a-0ab33e-openshift-default-65bc44c749 Created pod: e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt 42m Normal SuccessfulCreate replicaset/e2e-api-a-0ab33e-openshift-default-65bc44c749 Created pod: e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Created pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Created pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 42m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 6; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled down replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 10 to 6 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-6bb06a-openshift-default Scaled up replica set e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 from 1 to 4 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Created pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 42m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Stopping container istio-proxy 42m Normal AddedInterface pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt Add eth0 [10.134.0.112/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Deleted pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-8cn6j 42m Normal AddedInterface pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 Add eth0 [10.134.0.113/23] from ovn-kubernetes 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Deleted pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Deleted pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-w6qzt 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Deleted pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-6krwt 42m Normal Created pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt Container created 42m Normal Created pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 Container created 42m Normal Started pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 Container started 42m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 42m Normal ScalingReplicaSet deployment/e2e-api-b-433712-openshift-default Scaled up replica set e2e-api-b-433712-openshift-default-594dc9f747 from 0 to 1 42m Normal AddedLabel gateway/e2e-api-b-433712 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-433712 42m Normal Started pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt Container started 42m Normal ScalingReplicaSet deployment/e2e-inf-a-a558b3-openshift-default Scaled up replica set e2e-inf-a-a558b3-openshift-default-8476c499c5 from 0 to 1 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-f76fb5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-api-a-f76fb5-openshift-default-7755f59d9c Created pod: e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj 42m Normal SuccessfulCreate replicaset/e2e-api-a-f76fb5-openshift-default-7755f59d9c Created pod: e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr 42m Normal AddedLabel gateway/e2e-inf-a-a558b3 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-a558b3 42m Normal SuccessfulCreate replicaset/e2e-inf-a-a558b3-openshift-default-8476c499c5 Created pod: e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg 42m Normal SuccessfulCreate replicaset/e2e-api-a-f76fb5-openshift-default-7755f59d9c Created pod: e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr 42m Normal ScalingReplicaSet deployment/e2e-api-a-f76fb5-openshift-default Scaled up replica set e2e-api-a-f76fb5-openshift-default-7755f59d9c from 1 to 4 42m Warning ProbeError pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Readiness probe error: Get "http://10.133.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-84304f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Readiness probe error: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Readiness probe failed: Get "http://10.133.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-84304f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Readiness probe failed: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal Created pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Container created 42m Normal Started pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Container started 42m Warning Unhealthy pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Readiness probe failed: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused 42m Normal Created pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Container created 42m Normal Pulled pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal Started pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Container started 42m Normal Pulled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-jjn28 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Readiness probe error: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused... 42m Normal Pulled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Container created 42m Normal Started pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Container started 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-e526ec-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-e526ec-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Pulled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp Container created 42m Normal AddedInterface pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Add eth0 [10.133.0.177/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp Add eth0 [10.133.0.180/23] from ovn-kubernetes 42m Normal Created pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br Container created 42m Normal Pulled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp Container started 42m Normal Started pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br Container started 42m Normal Pulled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Container created 42m Normal AddedInterface pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 Add eth0 [10.133.0.178/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br Add eth0 [10.133.0.179/23] from ovn-kubernetes 42m Normal Started pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Container started 42m Normal Started pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 Container started 42m Normal Created pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 Container created 42m Normal SuccessfulCreate replicaset/e2e-api-b-c06e05-openshift-default-cfb8d84fc Created pod: e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp 42m Normal ScalingReplicaSet deployment/e2e-api-b-c06e05-openshift-default Scaled up replica set e2e-api-b-c06e05-openshift-default-cfb8d84fc from 0 to 1 42m Warning ProbeError pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 42m Normal AddedLabel gateway/e2e-api-b-c06e05 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-c06e05 42m Warning Unhealthy pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 42m Warning ProbeError pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Startup probe failed: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused 42m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Startup probe error: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Startup probe error: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Startup probe failed: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused 42m Warning ProbeError pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Startup probe error: Get "http://10.133.0.168:15021/healthz/ready": dial tcp 10.133.0.168:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Startup probe failed: Get "http://10.133.0.168:15021/healthz/ready": dial tcp 10.133.0.168:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 Startup probe failed: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused 42m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 Startup probe error: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused... 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Startup probe failed: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused 42m Warning ProbeError pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Startup probe error: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused... 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp Startup probe failed: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused 42m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp Startup probe error: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 42m Normal ScalingReplicaSet deployment/e2e-mt-d478b36e-openshift-default Scaled up replica set e2e-mt-d478b36e-openshift-default-56b9ccb7bf from 1 to 2 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-d478b36e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-0ab33e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-0ab33e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Deleted pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx 41m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Deleted pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Deleted pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-c28g9 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Deleted pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-rwpgm 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 6; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-28gxb Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled down replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 10 to 6 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-f76fb5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-f76fb5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 Add eth0 [10.133.0.184/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 Add eth0 [10.133.0.181/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-99whx Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 Container started 41m Normal AddedInterface pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp Add eth0 [10.133.0.182/23] from ovn-kubernetes 41m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 Container created 41m Warning Unhealthy pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Readiness probe failed: Get "http://10.133.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Readiness probe error: Get "http://10.133.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal SuccessfulCreate replicaset/e2e-api-a-1ae775-openshift-default-65f567689c Created pod: e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl 41m Normal ScalingReplicaSet deployment/e2e-api-a-1ae775-openshift-default Scaled up replica set e2e-api-a-1ae775-openshift-default-65f567689c from 0 to 1 41m Normal AddedLabel gateway/e2e-api-a-1ae775 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-1ae775 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-433712-69d496844c Created pod: payload-pre-processing-e2e-api-b-433712-69d496844c-sfn6r 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-433712-69d496844c Error creating: pods "payload-pre-processing-e2e-api-b-433712-69d496844c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-433712: serviceaccount "payload-processing-e2e-api-b-433712" not found 41m Normal ScalingReplicaSet deployment/e2e-inf-b-e224cd-openshift-default Scaled up replica set e2e-inf-b-e224cd-openshift-default-67657fc945 from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-inf-b-e224cd-openshift-default-67657fc945 Created pod: e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn 41m Warning FailedCreate replicaset/payload-processing-e2e-api-b-433712-7f84f4fdbd Error creating: pods "payload-processing-e2e-api-b-433712-7f84f4fdbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-433712: serviceaccount "payload-processing-e2e-api-b-433712" not found 41m Normal AddedLabel gateway/e2e-inf-b-e224cd Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-e224cd 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-433712 Scaled up replica set payload-pre-processing-e2e-api-b-433712-69d496844c from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-433712 Scaled up replica set payload-processing-e2e-api-b-433712-7f84f4fdbd from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-433712-7f84f4fdbd Created pod: payload-processing-e2e-api-b-433712-7f84f4fdbd-sdj56 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-433712-69d496844c-sfn6r Add eth0 [10.134.0.115/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-api-b-433712-69d496844c-sfn6r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-api-b-433712-69d496844c-sfn6r Container created 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-c06e05-6485ff7479 Error creating: pods "payload-pre-processing-e2e-api-b-c06e05-6485ff7479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-c06e05: serviceaccount "payload-processing-e2e-api-b-c06e05" not found 41m Normal Created pod/payload-processing-e2e-api-b-433712-7f84f4fdbd-sdj56 Container created 41m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 Startup probe error: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused... 41m Normal Started pod/payload-pre-processing-e2e-api-b-433712-69d496844c-sfn6r Container started 41m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 Startup probe failed: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-c06e05-6485ff7479 Created pod: payload-pre-processing-e2e-api-b-c06e05-6485ff7479-6mkhj 41m Normal Started pod/payload-processing-e2e-api-b-433712-7f84f4fdbd-sdj56 Container started 41m Normal Started pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Container started 41m Normal Pulled pod/payload-processing-e2e-api-b-433712-7f84f4fdbd-sdj56 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-api-b-433712-7f84f4fdbd-sdj56 Add eth0 [10.133.0.185/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Container created 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-c06e05 Scaled up replica set payload-pre-processing-e2e-api-b-c06e05-6485ff7479 from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-api-b-c06e05-fcc4d87f8 Error creating: pods "payload-processing-e2e-api-b-c06e05-fcc4d87f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-c06e05: serviceaccount "payload-processing-e2e-api-b-c06e05" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-c06e05-fcc4d87f8 Created pod: payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-c06e05 Scaled up replica set payload-processing-e2e-api-b-c06e05-fcc4d87f8 from 0 to 1 41m Normal Pulled pod/payload-pre-processing-e2e-api-b-c06e05-6485ff7479-6mkhj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled down replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 10 to 6 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 5; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled down replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 6 to 5 41m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-6bb06a-openshift-default Scaled up replica set e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 from 4 to 6 41m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default New size: 6; reason: All metrics below target 41m Normal Created pod/payload-pre-processing-e2e-api-b-c06e05-6485ff7479-6mkhj Container created 41m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Created pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Deleted pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Deleted pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Deleted pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Deleted pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b 41m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-lst5b Stopping container istio-proxy 41m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Created pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms 41m Normal Started pod/payload-pre-processing-e2e-api-b-c06e05-6485ff7479-6mkhj Container started 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Deleted pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-wnvlx 41m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Deleted pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Deleted pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-c06e05-6485ff7479-6mkhj Add eth0 [10.133.0.186/23] from ovn-kubernetes 41m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-xzzpg Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled down replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 10 to 6 41m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-lz69n Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-r25nk Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Deleted pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Deleted pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-bsp97 41m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-xbqjc Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-s2wp9 Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-api-a-f76fb5-openshift-default Scaled up replica set e2e-api-a-f76fb5-openshift-default-7755f59d9c from 4 to 7 41m Normal SuccessfulCreate replicaset/e2e-api-a-f76fb5-openshift-default-7755f59d9c Created pod: e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g 41m Normal SuccessfulCreate replicaset/e2e-api-a-f76fb5-openshift-default-7755f59d9c Created pod: e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m 41m Normal SuccessfulCreate replicaset/e2e-api-a-f76fb5-openshift-default-7755f59d9c Created pod: e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-f76fb5-openshift-default New size: 7; reason: All metrics below target 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a26426-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a26426-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning Unhealthy pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Startup probe failed: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Startup probe error: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused... 41m Normal AddedInterface pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms Add eth0 [10.134.0.116/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m Add eth0 [10.134.0.119/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh Add eth0 [10.134.0.118/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Readiness probe error: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Pulled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g Container started 41m Normal Pulled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn Add eth0 [10.134.0.121/23] from ovn-kubernetes 41m Normal Created pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g Container created 41m Normal AddedInterface pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g Add eth0 [10.132.0.113/23] from ovn-kubernetes 41m Normal Started pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms Container started 41m Normal Started pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Container started 41m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-c8dxk Readiness probe failed: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal AddedInterface pod/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 Add eth0 [10.134.0.122/23] from ovn-kubernetes 41m Normal Started pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh Container started 41m Normal Created pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh Container created 41m Normal Pulled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Add eth0 [10.134.0.117/23] from ovn-kubernetes 41m Normal Created pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m Container created 41m Normal Pulled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl Add eth0 [10.134.0.120/23] from ovn-kubernetes 41m Normal Created pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Container created 41m Normal Created pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms Container created 41m Normal Pulled pod/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 Container started 41m Normal Started pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m Container started 41m Normal Created pod/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 Container created 41m Normal AddedInterface pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Add eth0 [10.134.0.123/23] from ovn-kubernetes 41m Normal Started pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Container started 41m Normal Created pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Container created 41m Normal Killing pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n Stopping container payload-processing 41m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-1ae775 Scaled up replica set payload-processing-e2e-api-a-1ae775-5b67fb4bfd from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-1ae775 Scaled up replica set payload-pre-processing-e2e-api-a-1ae775-65879db4ff from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-api-a-1ae775-5b67fb4bfd Error creating: pods "payload-processing-e2e-api-a-1ae775-5b67fb4bfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-1ae775: serviceaccount "payload-processing-e2e-api-a-1ae775" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-1ae775-65879db4ff Created pod: payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-1ae775-65879db4ff Error creating: pods "payload-pre-processing-e2e-api-a-1ae775-65879db4ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-1ae775: serviceaccount "payload-processing-e2e-api-a-1ae775" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-1ae775-5b67fb4bfd Created pod: payload-processing-e2e-api-a-1ae775-5b67fb4bfd-2829w 41m Normal Pulling pod/payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d Add eth0 [10.132.0.114/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f Error creating: pods "payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e224cd: serviceaccount "payload-processing-e2e-inf-b-e224cd" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-e224cd Scaled up replica set payload-processing-e2e-inf-b-e224cd-5d647796cd from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-e224cd-5d647796cd Created pod: payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-e224cd-5d647796cd Error creating: pods "payload-processing-e2e-inf-b-e224cd-5d647796cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e224cd: serviceaccount "payload-processing-e2e-inf-b-e224cd" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-e224cd Scaled up replica set payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f from 0 to 1 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Deleted pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Deleted pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Deleted pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f Created pod: payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Deleted pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default New size: 6; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled down replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 10 to 6 41m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-l7wtq Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-m2xm2 Stopping container istio-proxy 41m Normal AddedInterface pod/payload-processing-e2e-api-a-1ae775-5b67fb4bfd-2829w Add eth0 [10.134.0.124/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d Container created 41m Normal Started pod/payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d Container started 41m Normal Pulled pod/payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.393s (3.393s including waiting). Image size: 125317101 bytes. 41m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-s8k47 Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused 41m Normal Pulled pod/payload-processing-e2e-api-a-1ae775-5b67fb4bfd-2829w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh Startup probe failed: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a26426-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh Startup probe error: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused... 41m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m Startup probe failed: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused 41m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m Startup probe error: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused... 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a26426-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused... 41m Normal Started pod/payload-processing-e2e-api-a-1ae775-5b67fb4bfd-2829w Container started 41m Normal Created pod/payload-processing-e2e-api-a-1ae775-5b67fb4bfd-2829w Container created 41m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Startup probe error: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Startup probe failed: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused 41m Normal Pulled pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-dkcd2 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal Started pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Container started 41m Normal Created pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Container created 41m Normal Pulled pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj Add eth0 [10.134.0.129/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf Add eth0 [10.134.0.127/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf Container created 41m Normal Started pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf Container started 41m Normal Created pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj Container created 41m Normal Pulled pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv Container created 41m Normal Created pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr Container created 41m Normal AddedInterface pod/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv Add eth0 [10.134.0.128/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv Container started 41m Normal Started pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr Container started 41m Normal AddedInterface pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr Add eth0 [10.134.0.126/23] from ovn-kubernetes 41m Normal Started pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj Container started 41m Normal AddedInterface pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Add eth0 [10.134.0.131/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Startup probe failed: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Startup probe error: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr Startup probe failed: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused 41m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr Startup probe error: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj Startup probe error: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj Startup probe failed: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-a558b3-688cbd55dd Created pod: payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7 Error creating: pods "payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a558b3: serviceaccount "payload-processing-e2e-inf-a-a558b3" not found 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-a558b3-688cbd55dd Error creating: pods "payload-processing-e2e-inf-a-a558b3-688cbd55dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a558b3: serviceaccount "payload-processing-e2e-inf-a-a558b3" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-a558b3 Scaled up replica set payload-processing-e2e-inf-a-a558b3-688cbd55dd from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-a558b3 Scaled up replica set payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7 Created pod: payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk 41m Normal Started pod/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk Add eth0 [10.134.0.134/23] from ovn-kubernetes 41m Warning Unhealthy pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-v9zrg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Normal Created pod/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk Container created 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-p468n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Normal Started pod/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf Container started 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf Add eth0 [10.134.0.135/23] from ovn-kubernetes 41m Normal Created pod/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf Container created 41m Normal Pulled pod/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled down replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 10 to 7 40m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Deleted pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-7l56c 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Deleted pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 40m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-fldg9 Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 7; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Deleted pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-zhxjt 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a558b3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a558b3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-e526ec-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-e526ec-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Deleted pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-4vrbl 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled down replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 7 to 6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-lr5vk 40m Normal ScalingReplicaSet deployment/e2e-api-b-433712-openshift-default Scaled up replica set e2e-api-b-433712-openshift-default-594dc9f747 from 1 to 4 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-433712-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-ddrwp 40m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-dsl47 40m Normal Started pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp Container started 40m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 Container created 40m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 Container started 40m Normal Created pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp Container created 40m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-c06e05-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-c06e05-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp Startup probe error: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp Startup probe failed: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-d478b36e Scaled up replica set payload-processing-e2e-mt-d478b36e-65b9f9c44b from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-mt-d478b36e-65b9f9c44b Error creating: pods "payload-processing-e2e-mt-d478b36e-65b9f9c44b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d478b36e: serviceaccount "payload-processing-e2e-mt-d478b36e" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-d478b36e Scaled up replica set payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5 Created pod: payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-276bc 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5 Error creating: pods "payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d478b36e: serviceaccount "payload-processing-e2e-mt-d478b36e" not found 40m Normal Created pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-276bc Container created 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-d478b36e-65b9f9c44b Created pod: payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 40m Normal AddedInterface pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-276bc Add eth0 [10.133.0.192/23] from ovn-kubernetes 40m Normal Pulled pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-276bc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-276bc Container started 39m Normal Killing pod/payload-pre-processing-e2e-api-a-0ab33e-5f5d8f6459-frxg9 Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-a-0ab33e-54f7dbfcc8-bsvsw Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-api-a-f76fb5-86dc79f6c9-6h728 Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-a-f76fb5-5d75b85b9d-64hcd Stopping container payload-processing 39m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 39m Normal ScalingReplicaSet deployment/e2e-api-b-63db1a-openshift-default Scaled up replica set e2e-api-b-63db1a-openshift-default-85c7894bbf from 0 to 1 39m Normal AddedLabel gateway/e2e-api-b-63db1a Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-63db1a 39m Normal AddedInterface pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 Add eth0 [10.134.0.140/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 Container started 39m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 Add eth0 [10.133.0.193/23] from ovn-kubernetes 39m Normal Created pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 Container created 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-433712-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-433712-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedCreate replicaset/payload-processing-e2e-api-b-63db1a-55c46bd5fd Error creating: pods "payload-processing-e2e-api-b-63db1a-55c46bd5fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-63db1a: serviceaccount "payload-processing-e2e-api-b-63db1a" not found 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-63db1a-674f496d67 Error creating: pods "payload-pre-processing-e2e-api-b-63db1a-674f496d67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-63db1a: serviceaccount "payload-processing-e2e-api-b-63db1a" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-63db1a-55c46bd5fd Created pod: payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-63db1a Scaled up replica set payload-pre-processing-e2e-api-b-63db1a-674f496d67 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-63db1a-674f496d67 Created pod: payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-63db1a Scaled up replica set payload-processing-e2e-api-b-63db1a-55c46bd5fd from 0 to 1 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-c06e05-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-c06e05-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning ProbeError pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Readiness probe error: Get "http://10.134.0.131:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Readiness probe failed: Get "http://10.134.0.131:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 39m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-5b5qc 39m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-zsz7m 39m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-z47b8 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-433712-openshift-default New size: 8; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-api-b-433712-openshift-default Scaled up replica set e2e-api-b-433712-openshift-default-594dc9f747 from 4 to 8 39m Normal SuccessfulCreate replicaset/e2e-inf-b-139eb7-openshift-default-77847fb8b7 Created pod: e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 39m Normal ScalingReplicaSet deployment/e2e-inf-b-139eb7-openshift-default Scaled up replica set e2e-inf-b-139eb7-openshift-default-77847fb8b7 from 0 to 1 39m Normal AddedLabel gateway/e2e-inf-b-139eb7 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-139eb7 39m Normal Pulled pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Pulled pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn Container created 39m Normal Started pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn Container started 39m Normal Started pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Container started 39m Normal Created pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Container created 39m Warning ProbeError pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn Startup probe failed: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn Startup probe error: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 39m Normal ScalingReplicaSet deployment/e2e-api-b-433712-openshift-default Scaled up replica set e2e-api-b-433712-openshift-default-594dc9f747 from 8 to 9 39m Normal SuccessfulCreate replicaset/e2e-api-b-433712-openshift-default-594dc9f747 Created pod: e2e-api-b-433712-openshift-default-594dc9f747-8wbcj 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-433712-openshift-default New size: 9; reason: All metrics below target 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-e224cd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-e224cd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a558b3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a558b3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-e224cd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-e224cd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 Add eth0 [10.132.0.116/23] from ovn-kubernetes 38m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 Container created 38m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 Container started 38m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 38m Normal Created pod/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 Container created 38m Normal Started pod/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb Container started 38m Normal Pulled pod/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 Add eth0 [10.132.0.117/23] from ovn-kubernetes 38m Normal AddedInterface pod/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb Add eth0 [10.132.0.118/23] from ovn-kubernetes 38m Normal Started pod/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 Container started 38m Normal Pulled pod/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb Container created 38m Normal Killing pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-2hdmt Stopping container istio-proxy 38m Normal Killing pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Stopping container istio-proxy 38m Normal Killing pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Stopping container istio-proxy 38m Normal Killing pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-72qh7 Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-qs62h Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-api-a-0ab33e-openshift-default-65bc44c749-9m9w5 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Killing pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Stopping container istio-proxy 38m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 Container created 38m Normal Killing pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tkndr Stopping container istio-proxy 38m Normal Killing pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-cw88m Stopping container istio-proxy 38m Normal Killing pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-wsqph Stopping container istio-proxy 38m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m Add eth0 [10.134.0.143/23] from ovn-kubernetes 38m Normal Killing pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-b2m5g Stopping container istio-proxy 38m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 Add eth0 [10.133.0.195/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Killing pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-r5hmj Stopping container istio-proxy 38m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 Container started 38m Normal Killing pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-dq9jh Stopping container istio-proxy 38m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 Container started 38m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 Add eth0 [10.134.0.144/23] from ovn-kubernetes 38m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 Container created 38m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m Container created 38m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m Container started 38m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning ProbeError pod/e2e-api-a-f76fb5-openshift-default-7755f59d9c-tclfr Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Killing pod/payload-pre-processing-e2e-api-b-433712-69d496844c-sfn6r Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-api-b-433712-7f84f4fdbd-sdj56 Stopping container payload-processing 38m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Container created 38m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused... 38m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Add eth0 [10.132.0.121/23] from ovn-kubernetes 38m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Container started 38m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 Add eth0 [10.132.0.122/23] from ovn-kubernetes 38m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused 38m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused... 38m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Container started 38m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp Container started 38m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Add eth0 [10.134.0.147/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp Add eth0 [10.134.0.148/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Add eth0 [10.134.0.146/23] from ovn-kubernetes 38m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Container created 38m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp Container created 38m Normal Started pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Container started 38m Normal Created pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Container created 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-139eb7-746465db5b Created pod: payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-139eb7-746465db5b Error creating: pods "payload-processing-e2e-inf-b-139eb7-746465db5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-139eb7: serviceaccount "payload-processing-e2e-inf-b-139eb7" not found 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-139eb7 Scaled up replica set payload-processing-e2e-inf-b-139eb7-746465db5b from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf Error creating: pods "payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-139eb7: serviceaccount "payload-processing-e2e-inf-b-139eb7" not found 38m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused... 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf Created pod: payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-139eb7 Scaled up replica set payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf from 0 to 1 38m Normal Killing pod/payload-pre-processing-e2e-api-b-c06e05-6485ff7479-6mkhj Stopping container payload-pre-processing 38m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled down replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 6 to 5 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Deleted pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 38m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-9rsx2 Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 5; reason: All metrics below target 38m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Readiness probe error: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Readiness probe failed: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Readiness probe error: Get "http://10.132.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Readiness probe failed: Get "http://10.132.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal Killing pod/payload-processing-e2e-api-b-c06e05-fcc4d87f8-mdhg6 Stopping container payload-processing 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-d478b36e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 38m Normal ScalingReplicaSet deployment/e2e-mt-d478b36e-openshift-default Scaled up replica set e2e-mt-d478b36e-openshift-default-56b9ccb7bf from 2 to 4 38m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d 38m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Readiness probe failed: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Readiness probe error: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg Add eth0 [10.134.0.150/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm Container started 38m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm Container created 38m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm Add eth0 [10.132.0.123/23] from ovn-kubernetes 38m Normal AddedInterface pod/payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh Add eth0 [10.134.0.149/23] from ovn-kubernetes 38m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 38m Normal Pulled pod/payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg Container created 38m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Startup probe failed: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Startup probe error: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 38m Normal Started pod/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg Container started 38m Normal Started pod/payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh Container started 38m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 38m Normal Created pod/payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh Container created 38m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Add eth0 [10.134.0.151/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Container created 38m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Container started 38m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm Startup probe error: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm Startup probe failed: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-63db1a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr 38m Normal ScalingReplicaSet deployment/e2e-api-b-63db1a-openshift-default Scaled up replica set e2e-api-b-63db1a-openshift-default-85c7894bbf from 1 to 4 38m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q 38m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p 38m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 Container started 38m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 Container created 38m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-d478b36e-openshift-default New size: 8; reason: 38m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl 38m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd 38m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k 38m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n 38m Normal ScalingReplicaSet deployment/e2e-mt-d478b36e-openshift-default Scaled up replica set e2e-mt-d478b36e-openshift-default-56b9ccb7bf from 4 to 8 38m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 Startup probe error: Get "http://10.133.0.193:15021/healthz/ready": dial tcp 10.133.0.193:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 Startup probe failed: Get "http://10.133.0.193:15021/healthz/ready": dial tcp 10.133.0.193:15021: connect: connection refused 37m Normal Killing pod/payload-pre-processing-e2e-api-a-1ae775-65879db4ff-k9q8d Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-api-a-1ae775-5b67fb4bfd-2829w Stopping container payload-processing 37m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Deleted pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-h5r7h 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 4; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled down replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 5 to 4 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-d478b36e-openshift-default New size: 10; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-mt-d478b36e-openshift-default Scaled up replica set e2e-mt-d478b36e-openshift-default-56b9ccb7bf from 8 to 10 37m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf (combined from similar events): Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l 37m Normal SuccessfulCreate replicaset/e2e-mt-d478b36e-openshift-default-56b9ccb7bf Created pod: e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj 37m Normal Started pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl Container started 37m Normal Created pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl Container created 37m Normal Pulled pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k Container started 37m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k Container created 37m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k Add eth0 [10.133.0.197/23] from ovn-kubernetes 37m Warning Unhealthy pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k Startup probe error: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused... 37m Normal ScalingReplicaSet deployment/e2e-inf-a-379a6d-openshift-default Scaled up replica set e2e-inf-a-379a6d-openshift-default-7969d87647 from 0 to 1 37m Normal AddedLabel gateway/e2e-inf-a-379a6d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-379a6d 37m Normal SuccessfulCreate replicaset/e2e-inf-a-379a6d-openshift-default-7969d87647 Created pod: e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k Startup probe failed: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled down replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 6 to 5 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled down replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 6 to 5 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default Scaled down replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 5 to 4 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 5; reason: All metrics below target 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 5; reason: All metrics below target 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Deleted pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Deleted pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Deleted pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn 37m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-jvv6j Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-pvvrw Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Deleted pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 5; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled down replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 6 to 5 37m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-5dhwt Stopping container istio-proxy 37m Normal Killing pod/payload-processing-e2e-mt-d478b36e-65b9f9c44b-wrgq4 Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-mt-d478b36e-6d476fb4f5-276bc Stopping container payload-pre-processing 37m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-xndxn Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal AddedInterface pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Add eth0 [10.134.0.153/23] from ovn-kubernetes 37m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj Add eth0 [10.132.0.124/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj Container created 37m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj Container started 37m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Add eth0 [10.132.0.125/23] from ovn-kubernetes 37m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Container created 37m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 37m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Container started 37m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-z6q2x 37m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 37m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-z6q2x Stopping container discovery 37m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Container created 37m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Container created 37m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Add eth0 [10.133.0.204/23] from ovn-kubernetes 37m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Add eth0 [10.133.0.201/23] from ovn-kubernetes 37m Normal Created pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Container created 37m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Container started 37m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Container started 37m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Add eth0 [10.133.0.202/23] from ovn-kubernetes 37m Normal Started pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Container started 37m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Pulled pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Add eth0 [10.133.0.200/23] from ovn-kubernetes 37m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Add eth0 [10.133.0.203/23] from ovn-kubernetes 37m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Container started 37m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Container created 37m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Container started 37m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Container created 37m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Container started 37m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Add eth0 [10.134.0.154/23] from ovn-kubernetes 37m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Container created 37m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-1ae775-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-1ae775-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-379a6d-5d6bb4d488 Error creating: pods "payload-processing-e2e-inf-a-379a6d-5d6bb4d488-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-379a6d: serviceaccount "payload-processing-e2e-inf-a-379a6d" not found 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-379a6d Scaled up replica set payload-processing-e2e-inf-a-379a6d-5d6bb4d488 from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-379a6d Scaled up replica set payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7 Error creating: pods "payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-379a6d: serviceaccount "payload-processing-e2e-inf-a-379a6d" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7 Created pod: payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-379a6d-5d6bb4d488 Created pod: payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-63db1a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-63db1a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 37m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Deleted pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Deleted pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-6bb06a-openshift-default Scaled down replica set e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 from 6 to 4 37m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs Stopping container istio-proxy 37m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Deleted pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-q9drs 37m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused... 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default New size: 3; reason: All metrics below target 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Startup probe failed: Get "http://10.133.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Startup probe error: Get "http://10.133.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Startup probe error: Get "http://10.133.0.204:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Startup probe failed: Get "http://10.133.0.204:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Startup probe error: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Startup probe error: Get "http://10.133.0.204:15021/healthz/ready": dial tcp 10.133.0.204:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Startup probe failed: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Startup probe failed: Get "http://10.133.0.204:15021/healthz/ready": dial tcp 10.133.0.204:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Startup probe failed: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Startup probe error: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused... 37m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 Liveness probe failed: dial tcp 10.133.0.160:50052: i/o timeout 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw Add eth0 [10.133.0.205/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Startup probe failed: Get "http://10.133.0.202:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Startup probe error: Get "http://10.133.0.202:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Startup probe error: Get "http://10.133.0.203:15021/healthz/ready": dial tcp 10.133.0.203:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Startup probe failed: Get "http://10.133.0.203:15021/healthz/ready": dial tcp 10.133.0.203:15021: connect: connection refused 37m Normal Started pod/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw Container started 37m Normal Started pod/payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl Container started 37m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused 37m Normal Created pod/payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl Container created 37m Normal Pulled pod/payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Killing pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-6k2br Stopping container istio-proxy 37m Normal Created pod/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw Container created 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl Add eth0 [10.133.0.206/23] from ovn-kubernetes 37m Normal Killing pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-p9hg6 Stopping container istio-proxy 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-1ae775-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-1ae775-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-p9rx9 Stopping container istio-proxy 37m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp 37m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp 37m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-xvvt4 Stopping container istio-proxy 37m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-63db1a-openshift-default New size: 8; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-api-b-63db1a-openshift-default Scaled up replica set e2e-api-b-63db1a-openshift-default-85c7894bbf from 4 to 8 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Stopping container istio-proxy 37m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp Container created 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-zsz7m Stopping container istio-proxy 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-z47b8 Stopping container istio-proxy 37m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp Add eth0 [10.134.0.156/23] from ovn-kubernetes 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-dsl47 Stopping container istio-proxy 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Stopping container istio-proxy 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Stopping container istio-proxy 37m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp Container started 37m Normal Killing pod/e2e-api-b-433712-openshift-default-594dc9f747-ddrwp Stopping container istio-proxy 37m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp Add eth0 [10.132.0.126/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled down replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 5 to 4 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 4; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled down replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 5 to 4 37m Normal AddedLabel gateway/e2e-inf-a-6bc7b2 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-6bc7b2 37m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf Container started 37m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld 37m Normal Killing pod/e2e-api-b-c06e05-openshift-default-cfb8d84fc-zb8gp Stopping container istio-proxy 37m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf Container created 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Deleted pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 37m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf Add eth0 [10.134.0.155/23] from ovn-kubernetes 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 4; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default Scaled down replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 5 to 4 37m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp Container created 37m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp Container started 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Deleted pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Deleted pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl 37m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 Add eth0 [10.134.0.157/23] from ovn-kubernetes 37m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-86fhl Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-inf-a-6bc7b2-openshift-default Scaled up replica set e2e-inf-a-6bc7b2-openshift-default-57d46999c5 from 0 to 1 37m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 Container created 37m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Stopping container istio-proxy 37m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-5b5qc Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Deleted pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk 37m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 Container started 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Deleted pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 37m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Add eth0 [10.133.0.207/23] from ovn-kubernetes 37m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-djcwk Stopping container istio-proxy 37m Normal AddedLabel gateway/e2e-inf-a-172c10 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-172c10 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled down replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 6 to 4 37m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulCreate replicaset/e2e-inf-a-172c10-openshift-default-6766bf6875 Created pod: e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x 37m Normal ScalingReplicaSet deployment/e2e-inf-a-172c10-openshift-default Scaled up replica set e2e-inf-a-172c10-openshift-default-6766bf6875 from 0 to 1 37m Normal AddedInterface pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Add eth0 [10.133.0.208/23] from ovn-kubernetes 37m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Readiness probe error: Get "http://10.133.0.104:15021/healthz/ready": read tcp 10.133.0.2:51142->10.133.0.104:15021: read: connection reset by peer... 37m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-fqmpt Readiness probe failed: Get "http://10.133.0.104:15021/healthz/ready": read tcp 10.133.0.2:51142->10.133.0.104:15021: read: connection reset by peer 37m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6hsq6 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b Created pod: payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt 37m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-6bc7b2 Scaled up replica set payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-6bc7b2 Scaled up replica set payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755 Error creating: pods "payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6bc7b2: serviceaccount "payload-processing-e2e-inf-a-6bc7b2" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755 Created pod: payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b Error creating: pods "payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6bc7b2: serviceaccount "payload-processing-e2e-inf-a-6bc7b2" not found 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt Add eth0 [10.133.0.209/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt Container started 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 Add eth0 [10.133.0.210/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 37m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-76rq9 Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 37m Normal Created pod/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt Container created 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-172c10 Scaled up replica set payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff Error creating: pods "payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-172c10: serviceaccount "payload-processing-e2e-inf-a-172c10" not found 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-172c10-57c45696d8 Error creating: pods "payload-processing-e2e-inf-a-172c10-57c45696d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-172c10: serviceaccount "payload-processing-e2e-inf-a-172c10" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff Created pod: payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd 36m Normal Created pod/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 Container created 36m Normal Started pod/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 Container started 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-172c10 Scaled up replica set payload-processing-e2e-inf-a-172c10-57c45696d8 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-172c10-57c45696d8 Created pod: payload-processing-e2e-inf-a-172c10-57c45696d8-9697h 36m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp Startup probe error: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused... 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd Add eth0 [10.133.0.211/23] from ovn-kubernetes 36m Warning Unhealthy pod/payload-processing-5f78687d5b-zt6f2 Readiness probe failed: timeout: failed to connect service "10.134.0.39:50052" within 1s: context deadline exceeded 36m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp Startup probe failed: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused 36m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Readiness probe error: Get "http://10.134.0.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-172c10-57c45696d8-9697h Add eth0 [10.133.0.212/23] from ovn-kubernetes 36m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Readiness probe failed: Get "http://10.134.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning Unhealthy pod/e2e-api-b-433712-openshift-default-594dc9f747-lr5vk Readiness probe failed: Get "http://10.134.0.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf Startup probe failed: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused 36m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf Startup probe error: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused... 36m Normal Created pod/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd Container created 36m Normal Started pod/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd Container started 36m Normal Pulled pod/payload-processing-e2e-inf-a-172c10-57c45696d8-9697h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 36m Normal Started pod/payload-processing-e2e-inf-a-172c10-57c45696d8-9697h Container started 36m Normal Created pod/payload-processing-e2e-inf-a-172c10-57c45696d8-9697h Container created 36m Warning ProbeError pod/e2e-api-b-433712-openshift-default-594dc9f747-8wbcj Readiness probe error: Get "http://10.134.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf (combined from similar events): Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-63db1a-openshift-default New size: 10; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-api-b-63db1a-openshift-default Scaled up replica set e2e-api-b-63db1a-openshift-default-85c7894bbf from 8 to 10 36m Normal SuccessfulCreate replicaset/e2e-api-b-63db1a-openshift-default-85c7894bbf Created pod: e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 36m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg Container created 36m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg Container started 36m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg Add eth0 [10.132.0.128/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 Container started 36m Normal Created pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 Container created 36m Normal AddedInterface pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 Add eth0 [10.132.0.127/23] from ovn-kubernetes 36m Normal Killing pod/e2e-api-a-1ae775-openshift-default-65f567689c-j6sbl Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 36m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 36m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 36m Normal Pulled pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 Container started 36m Normal Created pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 Container created 36m Normal Killing pod/payload-pre-processing-e2e-api-b-63db1a-674f496d67-mblf2 Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-api-b-63db1a-55c46bd5fd-2hxtb Stopping container payload-processing 36m Warning Unhealthy pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 Startup probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 Startup probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 36m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default Scaled down replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 4 to 3 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Deleted pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 36m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-w8789 Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 3; reason: All metrics below target 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-139eb7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-139eb7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-139eb7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-139eb7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-hwhbm Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-k928k Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Stopping container istio-proxy 36m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-xkm86 Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-72dnj Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-ffs2l Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 4; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default Scaled down replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 5 to 4 36m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Deleted pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-ztfwc 36m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-zv28d Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-5smjd Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-4lc2x Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-bn8dl Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Readiness probe failed: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/e2e-mt-d478b36e-openshift-default-56b9ccb7bf-88b8n Readiness probe error: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning ProbeError pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Readiness probe error: Get "http://10.134.0.153:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning Unhealthy pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Readiness probe failed: Get "http://10.134.0.153:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Normal Pulled pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Container started 35m Normal Created pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Container created 35m Normal SuccessfulCreate replicaset/e2e-inf-b-ad69b4-openshift-default-b754f7bb9 Created pod: e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr 35m Normal ScalingReplicaSet deployment/e2e-inf-b-ad69b4-openshift-default Scaled up replica set e2e-inf-b-ad69b4-openshift-default-b754f7bb9 from 0 to 1 35m Normal AddedLabel gateway/e2e-inf-b-ad69b4 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ad69b4 35m Normal AddedInterface pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Add eth0 [10.134.0.166/23] from ovn-kubernetes 35m Normal Killing pod/payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 Stopping container payload-pre-processing 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-437981-openshift-default Scaled up replica set e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc from 0 to 1 35m Normal AddedLabel gateway/e2e-shared-a-w5-437981 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-437981 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc Created pod: e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7 Created pod: payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8 Error creating: pods "payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ad69b4: serviceaccount "payload-processing-e2e-inf-b-ad69b4" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-ad69b4 Scaled up replica set payload-processing-e2e-inf-b-ad69b4-7dc6574fc8 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-ad69b4 Scaled up replica set payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7 from 0 to 1 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7 Error creating: pods "payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ad69b4: serviceaccount "payload-processing-e2e-inf-b-ad69b4" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8 Created pod: payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t 35m Normal Started pod/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz Container started 35m Normal Created pod/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz Add eth0 [10.133.0.228/23] from ovn-kubernetes 35m Normal AddedInterface pod/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t Add eth0 [10.133.0.229/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw Add eth0 [10.134.0.167/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz Container created 35m Normal Pulled pod/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Pulled pod/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 35m Normal Started pod/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t Container started 35m Warning Unhealthy pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-437981 Scaled up replica set payload-processing-e2e-shared-a-w5-437981-654b69f766 from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-437981-654b69f766 Error creating: pods "payload-processing-e2e-shared-a-w5-437981-654b69f766-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-437981: serviceaccount "payload-processing-e2e-shared-a-w5-437981" not found 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f Error creating: pods "payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-437981: serviceaccount "payload-processing-e2e-shared-a-w5-437981" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-437981 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f Created pod: payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 Add eth0 [10.134.0.168/23] from ovn-kubernetes 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-437981-654b69f766 Created pod: payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv 35m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 Container started 35m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv Add eth0 [10.134.0.169/23] from ovn-kubernetes 35m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Pulled pod/payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv Container created 35m Normal Started pod/payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv Container started 35m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 Container created 35m Normal Killing pod/e2e-inf-a-a26426-openshift-default-6766c9bbf-w6fnm Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-a558b3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal ScalingReplicaSet deployment/e2e-inf-a-a558b3-openshift-default Scaled up replica set e2e-inf-a-a558b3-openshift-default-8476c499c5 from 1 to 2 35m Normal SuccessfulCreate replicaset/e2e-inf-a-a558b3-openshift-default-8476c499c5 Created pod: e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 35m Normal AddedInterface pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 Add eth0 [10.133.0.231/23] from ovn-kubernetes 35m Normal Started pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 Container started 35m Normal Pulled pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 Container created 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-a26426-6dbb7f4b7-5fmf4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/payload-processing-e2e-inf-a-a26426-6b9b8c6db6-rfhzn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal SuccessfulCreate replicaset/e2e-inf-b-8ebc60-openshift-default-78b65df4d9 Created pod: e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 35m Normal Killing pod/payload-processing-e2e-inf-b-e224cd-5d647796cd-qvbzv Stopping container payload-processing 35m Normal ScalingReplicaSet deployment/e2e-inf-b-8ebc60-openshift-default Scaled up replica set e2e-inf-b-8ebc60-openshift-default-78b65df4d9 from 0 to 1 35m Normal AddedLabel gateway/e2e-inf-b-8ebc60 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-8ebc60 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf Stopping container payload-pre-processing 35m Normal AddedInterface pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Add eth0 [10.133.0.232/23] from ovn-kubernetes 35m Normal AddedLabel gateway/e2e-inf-b-8dd4c8 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-8dd4c8 35m Normal ScalingReplicaSet deployment/e2e-inf-b-8dd4c8-openshift-default Scaled up replica set e2e-inf-b-8dd4c8-openshift-default-7d77669699 from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-inf-b-8dd4c8-openshift-default-7d77669699 Created pod: e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv 35m Normal AddedInterface pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Add eth0 [10.133.0.233/23] from ovn-kubernetes 35m Warning Unhealthy pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 Startup probe failed: Get "http://10.133.0.231:15021/healthz/ready": dial tcp 10.133.0.231:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 Startup probe error: Get "http://10.133.0.231:15021/healthz/ready": dial tcp 10.133.0.231:15021: connect: connection refused... 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-379a6d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-379a6d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6bc7b2-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 35m Normal ScalingReplicaSet deployment/e2e-inf-a-6bc7b2-openshift-default Scaled up replica set e2e-inf-a-6bc7b2-openshift-default-57d46999c5 from 1 to 4 35m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft 35m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm 35m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Add eth0 [10.132.0.138/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-8ebc60 Scaled up replica set payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7 Created pod: payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7 Error creating: pods "payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8ebc60: serviceaccount "payload-processing-e2e-inf-b-8ebc60" not found 35m Normal Started pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Container started 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-8ebc60-58cbf955c8 Error creating: pods "payload-processing-e2e-inf-b-8ebc60-58cbf955c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8ebc60: serviceaccount "payload-processing-e2e-inf-b-8ebc60" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-8ebc60 Scaled up replica set payload-processing-e2e-inf-b-8ebc60-58cbf955c8 from 0 to 1 35m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Container created 35m Normal Pulled pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Container started 35m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Add eth0 [10.134.0.170/23] from ovn-kubernetes 35m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Container created 35m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Container created 35m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Container created 35m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Container started 35m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Container created 35m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Container started 35m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Container started 35m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Add eth0 [10.134.0.171/23] from ovn-kubernetes 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-8ebc60-58cbf955c8 Created pod: payload-processing-e2e-inf-b-8ebc60-58cbf955c8-7swqv 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-172c10-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-172c10-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c Error creating: pods "payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8dd4c8: serviceaccount "payload-processing-e2e-inf-b-8dd4c8" not found 35m Normal Started pod/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq Container started 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq Add eth0 [10.133.0.234/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/e2e-inf-b-139eb7-openshift-default Scaled up replica set e2e-inf-b-139eb7-openshift-default-77847fb8b7 from 1 to 2 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-437981-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-437981-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-139eb7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal SuccessfulCreate replicaset/e2e-inf-b-139eb7-openshift-default-77847fb8b7 Created pod: e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd 35m Normal Created pod/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq Container created 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-8dd4c8 Scaled up replica set payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-8dd4c8 Scaled up replica set payload-processing-e2e-inf-b-8dd4c8-bbd854d48 from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-8dd4c8-bbd854d48 Error creating: pods "payload-processing-e2e-inf-b-8dd4c8-bbd854d48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8dd4c8: serviceaccount "payload-processing-e2e-inf-b-8dd4c8" not found 35m Normal Pulled pod/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-processing-e2e-inf-b-8ebc60-58cbf955c8-7swqv Container started 35m Normal Created pod/payload-processing-e2e-inf-b-8ebc60-58cbf955c8-7swqv Container created 35m Normal Pulled pod/payload-processing-e2e-inf-b-8ebc60-58cbf955c8-7swqv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-inf-b-8ebc60-58cbf955c8-7swqv Add eth0 [10.133.0.235/23] from ovn-kubernetes 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-8dd4c8-bbd854d48 Created pod: payload-processing-e2e-inf-b-8dd4c8-bbd854d48-b8nkm 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c Created pod: payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx 35m Normal Pulled pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd Add eth0 [10.134.0.172/23] from ovn-kubernetes 35m Normal Pulled pod/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx Add eth0 [10.133.0.236/23] from ovn-kubernetes 35m Normal AddedInterface pod/payload-processing-e2e-inf-b-8dd4c8-bbd854d48-b8nkm Add eth0 [10.133.0.237/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-inf-b-8dd4c8-bbd854d48-b8nkm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-processing-e2e-inf-b-8dd4c8-bbd854d48-b8nkm Container started 35m Normal Created pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd Container created 35m Normal Created pod/payload-processing-e2e-inf-b-8dd4c8-bbd854d48-b8nkm Container created 35m Normal Started pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd Container started 35m Normal Created pod/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx Container created 35m Normal Started pod/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx Container started 35m Warning Unhealthy pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Startup probe failed: Get "http://10.133.0.208:15021/healthz/ready": dial tcp 10.133.0.208:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Startup probe error: Get "http://10.133.0.207:15021/healthz/ready": dial tcp 10.133.0.207:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Startup probe failed: Get "http://10.133.0.207:15021/healthz/ready": dial tcp 10.133.0.207:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Startup probe error: Get "http://10.133.0.208:15021/healthz/ready": dial tcp 10.133.0.208:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Startup probe error: Get "http://10.132.0.138:15021/healthz/ready": dial tcp 10.132.0.138:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Startup probe failed: Get "http://10.132.0.138:15021/healthz/ready": dial tcp 10.132.0.138:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Startup probe error: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Startup probe failed: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Startup probe failed: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-e224cd-5bfbb5879f-fcsrf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Startup probe error: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused... 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6bc7b2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-chw85 Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jztnp Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-6zczg Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-84kjp Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-p5nx4 Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-hgnvf Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Stopping container istio-proxy 35m Normal Killing pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-bjvk9 Stopping container istio-proxy 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6bc7b2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal AddedLabel gateway/e2e-inf-a-45e508 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-45e508 35m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal ScalingReplicaSet deployment/e2e-inf-a-45e508-openshift-default Scaled up replica set e2e-inf-a-45e508-openshift-default-585ccbd46f from 0 to 1 35m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-jtd6p Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal SuccessfulCreate replicaset/e2e-inf-a-45e508-openshift-default-585ccbd46f Created pod: e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 35m Normal AddedInterface pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Add eth0 [10.133.0.238/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-8b69q Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-api-b-63db1a-openshift-default-85c7894bbf-dlmxr Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-45e508-7b86876698 Error creating: pods "payload-processing-e2e-inf-a-45e508-7b86876698-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-45e508: serviceaccount "payload-processing-e2e-inf-a-45e508" not found 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f Error creating: pods "payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-45e508: serviceaccount "payload-processing-e2e-inf-a-45e508" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-45e508 Scaled up replica set payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f Created pod: payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-45e508 Scaled up replica set payload-processing-e2e-inf-a-45e508-7b86876698 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-45e508-7b86876698 Created pod: payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg Add eth0 [10.133.0.239/23] from ovn-kubernetes 34m Normal Created pod/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg Container created 34m Normal Pulled pod/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 Container started 34m Normal Created pod/payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 Container created 34m Normal Started pod/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg Container started 34m Normal Pulled pod/payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 Add eth0 [10.133.0.240/23] from ovn-kubernetes 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6bc7b2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6bc7b2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-172c10-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-172c10-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal ScalingReplicaSet deployment/e2e-inf-a-6bc7b2-openshift-default Scaled up replica set e2e-inf-a-6bc7b2-openshift-default-57d46999c5 from 4 to 8 34m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 Stopping container istio-proxy 34m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd 34m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6bc7b2-openshift-default New size: 8; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Deleted pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-wh4d7 34m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Deleted pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms 34m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs 34m Normal Killing pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-797ms Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default Scaled down replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 4 to 3 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default New size: 3; reason: All metrics below target 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 3; reason: All metrics below target 34m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-6bb06a-openshift-default Scaled down replica set e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 from 4 to 3 34m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Container started 34m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm Stopping container istio-proxy 34m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Container created 34m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Container started 34m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Stopping container istio-proxy 34m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Add eth0 [10.133.0.242/23] from ovn-kubernetes 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default New size: 3; reason: All metrics below target 34m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Add eth0 [10.133.0.241/23] from ovn-kubernetes 34m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Container created 34m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Container started 34m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Add eth0 [10.134.0.175/23] from ovn-kubernetes 34m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled down replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 4 to 3 34m Normal SuccessfulDelete replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Deleted pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b 34m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Add eth0 [10.132.0.139/23] from ovn-kubernetes 34m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default New size: 3; reason: All metrics below target 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-45e508-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-45e508-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled down replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 4 to 3 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 3; reason: All metrics below target 34m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Container created 34m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Container started 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default New size: 3; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Deleted pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 34m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Container created 34m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Deleted pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-vrjzm 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Deleted pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x 34m Normal Killing pod/e2e-inf-b-e224cd-openshift-default-67657fc945-d42qn Stopping container istio-proxy 34m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-bkz85 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-f2m6x Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Startup probe failed: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Startup probe error: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Readiness probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Startup probe error: Get "http://10.134.0.175:15021/healthz/ready": dial tcp 10.134.0.175:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nvx8b Readiness probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Startup probe error: Get "http://10.133.0.242:15021/healthz/ready": dial tcp 10.133.0.242:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Startup probe failed: Get "http://10.133.0.241:15021/healthz/ready": dial tcp 10.133.0.241:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Startup probe failed: Get "http://10.133.0.242:15021/healthz/ready": dial tcp 10.133.0.242:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Startup probe failed: Get "http://10.134.0.175:15021/healthz/ready": dial tcp 10.134.0.175:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Startup probe error: Get "http://10.133.0.241:15021/healthz/ready": dial tcp 10.133.0.241:15021: connect: connection refused... 34m Normal ScalingReplicaSet deployment/e2e-inf-a-6bc7b2-openshift-default Scaled up replica set e2e-inf-a-6bc7b2-openshift-default-57d46999c5 from 8 to 10 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6bc7b2-openshift-default New size: 10; reason: All metrics below target 34m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 (combined from similar events): Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 34m Normal SuccessfulCreate replicaset/e2e-inf-a-6bc7b2-openshift-default-57d46999c5 Created pod: e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 34m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Container created 34m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Container started 34m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Add eth0 [10.132.0.141/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Add eth0 [10.134.0.177/23] from ovn-kubernetes 34m Normal Started pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Container started 34m Normal Created pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Container created 34m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Startup probe error: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Startup probe failed: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Startup probe error: Get "http://10.134.0.177:15021/healthz/ready": dial tcp 10.134.0.177:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Startup probe failed: Get "http://10.134.0.177:15021/healthz/ready": dial tcp 10.134.0.177:15021: connect: connection refused 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf Created pod: e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw 34m Normal AddedLabel gateway/e2e-shared-b-w5-dbb507 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-dbb507 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-dbb507-openshift-default Scaled up replica set e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf from 0 to 1 34m Normal AddedInterface pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Add eth0 [10.134.0.178/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664 Created pod: payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-dbb507 Scaled up replica set payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664 from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-dbb507 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59 from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59 Created pod: payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 33m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664 Error creating: pods "payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-dbb507: serviceaccount "payload-processing-e2e-shared-b-w5-dbb507" not found 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-dbb507: serviceaccount "payload-processing-e2e-shared-b-w5-dbb507" not found 33m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 Container created 33m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 Container started 33m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 Add eth0 [10.133.0.245/23] from ovn-kubernetes 33m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf Add eth0 [10.134.0.179/23] from ovn-kubernetes 33m Normal Pulled pod/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf Container created 33m Normal Started pod/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf Container started 33m Warning Unhealthy pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Readiness probe failed: Get "http://10.133.0.232:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Readiness probe error: Get "http://10.133.0.232:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning ProbeError pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Readiness probe error: Get "http://10.133.0.233:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Readiness probe failed: Get "http://10.133.0.233:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Readiness probe error: Get "http://10.134.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Normal Pulled pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning Unhealthy pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Readiness probe failed: Get "http://10.134.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Normal Pulled pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Container created 33m Normal Started pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Container started 33m Normal Started pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Container started 33m Normal Created pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Container created 33m Normal Pulled pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Container created 33m Normal Started pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Container started 33m Normal Created pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw Container created 33m Normal Pulled pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw Container started 33m Warning ProbeError pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Startup probe error: Get "http://10.133.0.232:15021/healthz/ready": dial tcp 10.133.0.232:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Startup probe failed: Get "http://10.133.0.232:15021/healthz/ready": dial tcp 10.133.0.232:15021: connect: connection refused 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8ebc60-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8ebc60-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Startup probe failed: Get "http://10.134.0.166:15021/healthz/ready": dial tcp 10.134.0.166:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Startup probe error: Get "http://10.134.0.166:15021/healthz/ready": dial tcp 10.134.0.166:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Startup probe failed: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Startup probe error: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused 33m Normal SuccessfulCreate replicaset/e2e-inf-b-139eb7-openshift-default-77847fb8b7 Created pod: e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd 33m Normal ScalingReplicaSet deployment/e2e-inf-b-139eb7-openshift-default Scaled up replica set e2e-inf-b-139eb7-openshift-default-77847fb8b7 from 2 to 3 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-139eb7-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 33m Normal Started pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd Container started 33m Normal AddedInterface pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd Add eth0 [10.133.0.248/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd Container created 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c Created pod: e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-afdbe8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-afdbe8-openshift-default Scaled up replica set e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c from 1 to 2 33m Normal Pulled pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 Container created 33m Normal Started pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 Container started 33m Normal AddedInterface pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 Add eth0 [10.132.0.143/23] from ovn-kubernetes 33m Warning Unhealthy pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd Startup probe failed: Get "http://10.133.0.248:15021/healthz/ready": dial tcp 10.133.0.248:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd Startup probe error: Get "http://10.133.0.248:15021/healthz/ready": dial tcp 10.133.0.248:15021: connect: connection refused... 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8dd4c8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8dd4c8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal AddedLabel gateway/e2e-inf-b-57d336 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-57d336 33m Normal ScalingReplicaSet deployment/e2e-inf-b-57d336-openshift-default Scaled up replica set e2e-inf-b-57d336-openshift-default-854545cc57 from 0 to 1 33m Normal SuccessfulCreate replicaset/e2e-inf-b-57d336-openshift-default-854545cc57 Created pod: e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l 33m Normal AddedInterface pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Add eth0 [10.133.0.249/23] from ovn-kubernetes 33m Warning ProbeError pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-437981-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-437981-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-57d336 Scaled up replica set payload-processing-e2e-inf-b-57d336-5f55d66b86 from 0 to 1 33m Warning ProbeError pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Readiness probe error: Get "http://10.133.0.238:15021/healthz/ready": read tcp 10.133.0.2:47580->10.133.0.238:15021: read: connection reset by peer... 33m Warning Unhealthy pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Readiness probe failed: Get "http://10.133.0.238:15021/healthz/ready": read tcp 10.133.0.2:47580->10.133.0.238:15021: read: connection reset by peer 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-57d336 Scaled up replica set payload-pre-processing-e2e-inf-b-57d336-6f75d45b47 from 0 to 1 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47 Error creating: pods "payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-57d336: serviceaccount "payload-processing-e2e-inf-b-57d336" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47 Created pod: payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-57d336-5f55d66b86 Created pod: payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-57d336-5f55d66b86 Error creating: pods "payload-processing-e2e-inf-b-57d336-5f55d66b86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-57d336: serviceaccount "payload-processing-e2e-inf-b-57d336" not found 33m Normal Started pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Container started 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 Add eth0 [10.134.0.181/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Container created 33m Normal Pulled pod/payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw Add eth0 [10.134.0.182/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 Container started 33m Normal Created pod/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 Container created 33m Normal Pulled pod/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Started pod/payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw Container started 33m Normal Created pod/payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw Container created 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8ebc60-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8ebc60-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning ProbeError pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Startup probe error: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Startup probe failed: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ad69b4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8dd4c8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8dd4c8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ad69b4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal ScalingReplicaSet deployment/e2e-worker-w6-92fd86-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-92fd86-openshift-default-5c74588f5 from 3 to 2 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-92fd86-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 33m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/e2e-worker-w6-92fd86-openshift-default-5c74588f5 Deleted pod: e2e-worker-w6-92fd86-openshift-default-5c74588f5-gsqxd 32m Normal Killing pod/payload-processing-e2e-inf-a-a558b3-688cbd55dd-95wlf Stopping container payload-processing 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-45e508-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk Stopping container payload-pre-processing 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-45e508-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Killing pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-jjjvg Stopping container istio-proxy 32m Normal Killing pod/e2e-inf-a-a558b3-openshift-default-8476c499c5-4vmt4 Stopping container istio-proxy 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-a558b3-d6dc777b7-xxfvk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg Stopping container payload-pre-processing 32m Normal SuccessfulCreate replicaset/e2e-inf-b-ad69b4-openshift-default-b754f7bb9 Created pod: e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn 32m Normal ScalingReplicaSet deployment/e2e-inf-b-ad69b4-openshift-default Scaled up replica set e2e-inf-b-ad69b4-openshift-default-b754f7bb9 from 1 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-ad69b4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-379a6d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal AddedInterface pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn Add eth0 [10.133.0.253/23] from ovn-kubernetes 32m Normal Started pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn Container started 32m Normal Created pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn Container created 32m Normal Pulled pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal SuccessfulCreate replicaset/e2e-inf-a-379a6d-openshift-default-7969d87647 Created pod: e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp 32m Normal ScalingReplicaSet deployment/e2e-inf-a-379a6d-openshift-default Scaled up replica set e2e-inf-a-379a6d-openshift-default-7969d87647 from 1 to 2 32m Normal Pulled pod/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp Container started 32m Normal Created pod/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp Container created 32m Normal AddedInterface pod/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp Add eth0 [10.132.0.146/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn Startup probe failed: Get "http://10.133.0.253:15021/healthz/ready": dial tcp 10.133.0.253:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn Startup probe error: Get "http://10.133.0.253:15021/healthz/ready": dial tcp 10.133.0.253:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused 32m Normal Created pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Container created 32m Normal Pulled pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Container started 32m Warning Unhealthy pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Startup probe failed: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Startup probe error: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused... 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-dbb507-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-dbb507-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-84304f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal Killing pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-nllf5 Stopping container istio-proxy 32m Normal Killing pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-k97cd Stopping container istio-proxy 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 Created pod: e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r 32m Normal Killing pod/e2e-inf-b-139eb7-openshift-default-77847fb8b7-ckcwd Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-84304f-openshift-default Scaled up replica set e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 from 1 to 2 32m Normal Pulled pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Container created 32m Normal Started pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Container started 32m Normal AddedInterface pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Add eth0 [10.134.0.188/23] from ovn-kubernetes 32m Warning Unhealthy pod/payload-processing-e2e-inf-b-139eb7-746465db5b-7l9zh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-139eb7-56594ffbcf-grqkg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning ProbeError pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Startup probe error: Get "http://10.134.0.188:15021/healthz/ready": dial tcp 10.134.0.188:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Startup probe failed: Get "http://10.134.0.188:15021/healthz/ready": dial tcp 10.134.0.188:15021: connect: connection refused 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-dbb507-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-dbb507-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning Unhealthy pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Readiness probe failed: Get "http://10.133.0.249:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 31m Warning ProbeError pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Readiness probe error: Get "http://10.133.0.249:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 31m Normal Created pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Container created 31m Normal Pulled pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Container started 31m Warning ProbeError pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Startup probe error: Get "http://10.133.0.249:15021/healthz/ready": dial tcp 10.133.0.249:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Startup probe failed: Get "http://10.133.0.249:15021/healthz/ready": dial tcp 10.133.0.249:15021: connect: connection refused 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-57d336-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-57d336-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-437981-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-437981-openshift-default Scaled up replica set e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc from 1 to 2 31m Normal Pulled pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Container started 31m Normal AddedInterface pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Add eth0 [10.133.0.254/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc Created pod: e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv 31m Normal Created pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Container created 31m Warning Unhealthy pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Startup probe failed: Get "http://10.133.0.254:15021/healthz/ready": dial tcp 10.133.0.254:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Startup probe error: Get "http://10.133.0.254:15021/healthz/ready": dial tcp 10.133.0.254:15021: connect: connection refused... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-57d336-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-57d336-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-45e508-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-inf-a-45e508-openshift-default Scaled up replica set e2e-inf-a-45e508-openshift-default-585ccbd46f from 1 to 2 30m Normal SuccessfulCreate replicaset/e2e-inf-a-45e508-openshift-default-585ccbd46f Created pod: e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk 30m Normal Pulled pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk Container started 30m Normal AddedInterface pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk Add eth0 [10.134.0.191/23] from ovn-kubernetes 30m Normal SuccessfulCreate replicaset/e2e-inf-a-172c10-openshift-default-6766bf6875 Created pod: e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 30m Normal Created pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk Container created 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-172c10-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-inf-a-172c10-openshift-default Scaled up replica set e2e-inf-a-172c10-openshift-default-6766bf6875 from 1 to 2 30m Normal AddedInterface pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Add eth0 [10.134.0.192/23] from ovn-kubernetes 30m Normal Started pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Container started 30m Normal Pulled pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Container created 30m Warning ProbeError pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk Startup probe error: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk Startup probe failed: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Startup probe failed: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Startup probe error: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused... 30m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Stopping container istio-proxy 30m Normal ScalingReplicaSet deployment/e2e-worker-w4-4a0287-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 from 3 to 2 30m Normal SuccessfulDelete replicaset/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5 Deleted pod: e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 2; reason: All metrics below target 30m Normal Killing pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Stopping container istio-proxy 30m Normal SuccessfulDelete replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Deleted pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4a0287-openshift-default New size: 2; reason: All metrics below target 30m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Deleted pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q 30m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default Scaled down replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 3 to 2 30m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Stopping container istio-proxy 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-879596-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-95672c-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 30m Normal SuccessfulDelete replicaset/e2e-worker-w3-95672c-openshift-default-5774db9bc Deleted pod: e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr 30m Normal ScalingReplicaSet deployment/e2e-worker-w3-95672c-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-95672c-openshift-default-5774db9bc from 3 to 2 30m Normal SuccessfulDelete replicaset/e2e-worker-w1-879596-openshift-default-6b7d57dcc5 Deleted pod: e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 30m Normal ScalingReplicaSet deployment/e2e-worker-w1-879596-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-879596-openshift-default-6b7d57dcc5 from 3 to 2 30m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-jsggr Stopping container istio-proxy 30m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Readiness probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-2nk7q Readiness probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Readiness probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 30m Warning ProbeError pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-gxp2t Readiness probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Readiness probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-5cqz7 Readiness probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-jjc45 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal SuccessfulCreate replicaset/e2e-inf-b-8ebc60-openshift-default-78b65df4d9 Created pod: e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-8ebc60-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal ScalingReplicaSet deployment/e2e-inf-b-8ebc60-openshift-default Scaled up replica set e2e-inf-b-8ebc60-openshift-default-78b65df4d9 from 1 to 2 29m Normal Started pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg Container started 29m Normal Created pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg Container created 29m Normal Pulled pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg Add eth0 [10.132.0.149/23] from ovn-kubernetes 29m Warning Unhealthy pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused 29m Warning ProbeError pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused... 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Created pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-6bb06a-openshift-default Scaled up replica set e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 from 2 to 3 29m Normal ScalingReplicaSet deployment/e2e-worker-w5-0d12c5-openshift-default Scaled down replica set e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 from 3 to 2 29m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7 Deleted pod: e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq 29m Normal AddedInterface pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v Add eth0 [10.134.0.193/23] from ovn-kubernetes 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-0d12c5-openshift-default New size: 2; reason: All metrics below target 29m Normal Pulled pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v Container started 29m Normal Created pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v Container created 29m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v Startup probe error: Get "http://10.134.0.193:15021/healthz/ready": dial tcp 10.134.0.193:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-6b5qq Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v Startup probe failed: Get "http://10.134.0.193:15021/healthz/ready": dial tcp 10.134.0.193:15021: connect: connection refused 29m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-st7pn Stopping container discovery 29m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-st7pn 29m Normal Killing pod/payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl Stopping container payload-processing 29m Normal Killing pod/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 Stopping container payload-pre-processing 29m Normal Killing pod/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw Stopping container payload-pre-processing 29m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 10 to 7 29m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Deleted pod: maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Deleted pod: maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Deleted pod: maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 29m Normal Killing pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Stopping container istio-proxy 29m Normal Killing pod/maas-default-gateway-openshift-default-6f77ddbcb7-x4fl9 Stopping container istio-proxy 29m Normal Killing pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Stopping container istio-proxy 29m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zbv7 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-pqjzr Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc Created pod: e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-437981-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Normal AddedInterface pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Add eth0 [10.132.0.151/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-437981-openshift-default Scaled up replica set e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc from 2 to 3 29m Normal Pulled pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Container started 29m Normal Created pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Container created 28m Warning Unhealthy pod/payload-processing-e2e-inf-a-6bc7b2-7cdc874c7b-jr2xt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/e2e-inf-a-379a6d-openshift-default-7969d87647-zjp5b Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-379a6d-openshift-default-7969d87647-hzmhp Stopping container istio-proxy 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-379a6d-7487dfc7d7-xqnsw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning ProbeError pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Startup probe error: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused... 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-6bc7b2-689ff5c755-rms74 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Startup probe failed: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused 28m Warning Unhealthy pod/payload-processing-e2e-inf-a-379a6d-5d6bb4d488-59zfl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz Stopping container payload-pre-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-a-172c10-57c45696d8-9697h Stopping container payload-processing 28m Normal ScalingReplicaSet deployment/e2e-worker-w2-8d3632-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 from 3 to 2 28m Normal SuccessfulDelete replicaset/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9 Deleted pod: e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf 28m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-rkzgf Stopping container istio-proxy 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-8d3632-openshift-default New size: 2; reason: All metrics below target 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Stopping container istio-proxy 28m Warning ProbeError pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-q4h54 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-jpmft Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal Killing pod/e2e-inf-a-172c10-openshift-default-6766bf6875-zm58x Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-tjnt4 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal Killing pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Stopping container istio-proxy 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-txjld Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-nbwcd Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-sdnhg Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-qqwzs Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-765tk Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-2lngm Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning ProbeError pod/e2e-inf-a-6bc7b2-openshift-default-57d46999c5-g5x9p Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-ad69b4-6c5649b4b7-bkmwz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-processing-e2e-inf-b-ad69b4-7dc6574fc8-8tp7t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-processing-e2e-inf-a-172c10-57c45696d8-9697h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning ProbeError pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-172c10-6b9f9f49ff-2tsmd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/e2e-inf-a-172c10-openshift-default-6766bf6875-rr2l8 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal Killing pod/payload-processing-e2e-inf-b-8dd4c8-bbd854d48-b8nkm Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx Stopping container payload-pre-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-b-8ebc60-58cbf955c8-7swqv Stopping container payload-processing 28m Normal Killing pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-2tlcn Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-ad69b4-openshift-default-b754f7bb9-9nmrr Stopping container istio-proxy 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-f7a5fe-openshift-default-b997867f9 Created pod: e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-f7a5fe-openshift-default Scaled up replica set e2e-ipp-a-f7a5fe-openshift-default-b997867f9 from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-a-f7a5fe Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f7a5fe 28m Normal SuccessfulCreate replicaset/e2e-inf-b-57d336-openshift-default-854545cc57 Created pod: e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-57d336-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-inf-b-57d336-openshift-default Scaled up replica set e2e-inf-b-57d336-openshift-default-854545cc57 from 1 to 2 28m Normal AddedInterface pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Add eth0 [10.134.0.194/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn Add eth0 [10.134.0.195/23] from ovn-kubernetes 28m Normal Started pod/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn Container started 28m Normal Created pod/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn Container created 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-dbb507-openshift-default Scaled up replica set e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf from 1 to 2 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-dbb507-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf Created pod: e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp 28m Normal Created pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp Container created 28m Normal Started pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp Container started 28m Normal Pulled pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp Add eth0 [10.132.0.153/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f7a5fe-c864778c6 Created pod: payload-processing-e2e-ipp-a-f7a5fe-c864778c6-vw2dk 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f7a5fe-c864778c6 Error creating: pods "payload-processing-e2e-ipp-a-f7a5fe-c864778c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f7a5fe: serviceaccount "payload-processing-e2e-ipp-a-f7a5fe" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f7a5fe Scaled up replica set payload-processing-e2e-ipp-a-f7a5fe-c864778c6 from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f7a5fe Scaled up replica set payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c Created pod: payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-gpwpk 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c Error creating: pods "payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f7a5fe: serviceaccount "payload-processing-e2e-ipp-a-f7a5fe" not found 28m Normal Started pod/payload-processing-e2e-ipp-a-f7a5fe-c864778c6-vw2dk Container started 28m Normal Created pod/payload-processing-e2e-ipp-a-f7a5fe-c864778c6-vw2dk Container created 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-gpwpk Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-gpwpk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-gpwpk Add eth0 [10.134.0.196/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f7a5fe-c864778c6-vw2dk Add eth0 [10.134.0.197/23] from ovn-kubernetes 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-gpwpk Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-a-f7a5fe-c864778c6-vw2dk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn Startup probe error: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn Startup probe failed: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-8ebc60-78bc6d6bb7-gqdzq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning ProbeError pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp Startup probe error: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp Startup probe failed: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-8dd4c8-b9f49cd6c-wr4nx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-8xlkg Stopping container istio-proxy 27m Normal AddedLabel gateway/e2e-shared-a-w4-9a15f4 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-9a15f4 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9a15f4-openshift-default Scaled up replica set e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc Created pod: e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln 27m Normal Killing pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Stopping container istio-proxy 27m Normal AddedInterface pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln Add eth0 [10.134.0.198/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-329f7b-openshift-default Scaled up replica set e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 from 0 to 1 27m Normal AddedLabel gateway/e2e-shared-a-w1-329f7b Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-329f7b 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 Created pod: e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 27m Normal AddedInterface pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Add eth0 [10.134.0.199/23] from ovn-kubernetes 27m Warning Unhealthy pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-inf-b-8ebc60-openshift-default-78b65df4d9-sw624 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning ProbeError pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-inf-b-8dd4c8-openshift-default-7d77669699-w5dfv Readiness probe failed: HTTP probe failed with statuscode: 503 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-9a15f4 Scaled up replica set payload-processing-e2e-shared-a-w4-9a15f4-779b664b68 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68 Created pod: payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq 27m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68 Error creating: pods "payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-9a15f4: serviceaccount "payload-processing-e2e-shared-a-w4-9a15f4" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5 Created pod: payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-9a15f4: serviceaccount "payload-processing-e2e-shared-a-w4-9a15f4" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-9a15f4 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5 from 0 to 1 27m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq Add eth0 [10.134.0.200/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc Add eth0 [10.134.0.201/23] from ovn-kubernetes 27m Normal Started pod/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq Container started 27m Normal Created pod/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq Container created 27m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc Container created 27m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc Container started 27m Normal Pulled pod/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-329f7b Scaled up replica set payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-329f7b Scaled up replica set payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff from 0 to 1 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff Error creating: pods "payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-329f7b: serviceaccount "payload-processing-e2e-shared-a-w1-329f7b" not found 27m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb Error creating: pods "payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-329f7b: serviceaccount "payload-processing-e2e-shared-a-w1-329f7b" not found 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb Created pod: payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff Created pod: payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n 27m Normal Pulled pod/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p Add eth0 [10.134.0.203/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n Add eth0 [10.134.0.202/23] from ovn-kubernetes 27m Normal Started pod/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p Container started 27m Normal Created pod/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p Container created 27m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n Container started 27m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n Container created 27m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Deleted pod: maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr 27m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Deleted pod: maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s 27m Normal Killing pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Stopping container istio-proxy 27m Normal Killing pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 7 to 5 27m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 27m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-nmx8s Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Readiness probe failed: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused 27m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-q5zwr Readiness probe error: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused... 26m Normal Killing pod/payload-processing-e2e-ipp-a-f7a5fe-c864778c6-vw2dk Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5c8b67846c-gpwpk Stopping container payload-pre-processing 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-09fc89-openshift-default-869878cbf5 Created pod: e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp 26m Normal AddedLabel gateway/e2e-ipp-b-09fc89 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-09fc89 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-09fc89-openshift-default Scaled up replica set e2e-ipp-b-09fc89-openshift-default-869878cbf5 from 0 to 1 26m Normal AddedInterface pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Add eth0 [10.133.1.23/23] from ovn-kubernetes 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-09fc89-695597ffd8 Created pod: payload-processing-e2e-ipp-b-09fc89-695597ffd8-bntdb 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75 Created pod: payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-csl2b 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75 Error creating: pods "payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-09fc89: serviceaccount "payload-processing-e2e-ipp-b-09fc89" not found 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-09fc89-695597ffd8 Error creating: pods "payload-processing-e2e-ipp-b-09fc89-695597ffd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-09fc89: serviceaccount "payload-processing-e2e-ipp-b-09fc89" not found 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-09fc89 Scaled up replica set payload-processing-e2e-ipp-b-09fc89-695597ffd8 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-09fc89 Scaled up replica set payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75 from 0 to 1 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-csl2b Container started 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-csl2b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-csl2b Add eth0 [10.133.1.24/23] from ovn-kubernetes 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-09fc89-695597ffd8-bntdb Add eth0 [10.134.0.206/23] from ovn-kubernetes 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-csl2b Container created 26m Normal Started pod/payload-processing-e2e-ipp-b-09fc89-695597ffd8-bntdb Container started 26m Normal Created pod/payload-processing-e2e-ipp-b-09fc89-695597ffd8-bntdb Container created 26m Normal Pulled pod/payload-processing-e2e-ipp-b-09fc89-695597ffd8-bntdb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-e81b56-openshift-default Scaled up replica set e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69 from 1 to 2 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-e81b56-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69 Created pod: e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 26m Normal AddedInterface pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 Add eth0 [10.134.0.207/23] from ovn-kubernetes 26m Normal Created pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Container created 26m Normal Started pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Container started 26m Normal Pulled pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Pulled pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 Container created 26m Normal Started pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 Container started 26m Normal Pulled pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln Container created 26m Normal Started pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln Container started 26m Warning ProbeError pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 Startup probe error: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 Startup probe failed: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused 26m Warning Unhealthy pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln Startup probe failed: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln Startup probe error: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused... 26m Normal AddedLabel gateway/e2e-shared-b-w4-2ae845 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-2ae845 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-2ae845-openshift-default Scaled up replica set e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf from 0 to 1 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf Created pod: e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq 26m Normal AddedInterface pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Add eth0 [10.133.1.30/23] from ovn-kubernetes 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c Created pod: e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b 26m Normal AddedLabel gateway/e2e-shared-b-w1-e803b6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-e803b6 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 Created pod: e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e803b6-openshift-default Scaled up replica set e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c from 0 to 1 26m Normal AddedInterface pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Add eth0 [10.132.0.158/23] from ovn-kubernetes 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-84304f-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-84304f-openshift-default Scaled up replica set e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 from 2 to 3 26m Normal Pulled pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Add eth0 [10.134.0.208/23] from ovn-kubernetes 26m Normal Started pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Container started 26m Normal Created pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Container created 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-9a15f4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-9a15f4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-2ae845-748796f66d Created pod: payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-2ae845 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87 Created pod: payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd 26m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-2ae845-748796f66d Error creating: pods "payload-processing-e2e-shared-b-w4-2ae845-748796f66d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-2ae845: serviceaccount "payload-processing-e2e-shared-b-w4-2ae845" not found 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-2ae845: serviceaccount "payload-processing-e2e-shared-b-w4-2ae845" not found 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-2ae845 Scaled up replica set payload-processing-e2e-shared-b-w4-2ae845-748796f66d from 0 to 1 26m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd Add eth0 [10.134.0.210/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd Container started 26m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn Add eth0 [10.134.0.209/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Pulled pod/payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn Container created 26m Normal Started pod/payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn Container started 26m Normal Killing pod/payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 Stopping container payload-processing 26m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd Container created 26m Normal Killing pod/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg Stopping container payload-pre-processing 26m Warning Unhealthy pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Startup probe failed: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Startup probe error: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused... 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-e803b6: serviceaccount "payload-processing-e2e-shared-b-w1-e803b6" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb Created pod: payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-e803b6 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5 Created pod: payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-e803b6 Scaled up replica set payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb Error creating: pods "payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-e803b6: serviceaccount "payload-processing-e2e-shared-b-w1-e803b6" not found 26m Normal Created pod/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 Container created 26m Normal Started pod/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 Container started 26m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn Container started 26m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn Container created 26m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn Add eth0 [10.133.1.32/23] from ovn-kubernetes 26m Normal Pulled pod/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 Add eth0 [10.133.1.33/23] from ovn-kubernetes 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f7a5fe-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f7a5fe-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f7a5fe Scaled up replica set payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794 Error creating: pods "payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f7a5fe: serviceaccount "payload-processing-e2e-ipp-a-f7a5fe" not found 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f7a5fe Scaled up replica set payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c from 0 to 1 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c Error creating: pods "payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f7a5fe: serviceaccount "payload-processing-e2e-ipp-a-f7a5fe" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c Created pod: payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw Add eth0 [10.134.0.211/23] from ovn-kubernetes 26m Normal Pulling pod/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw Add eth0 [10.132.0.159/23] from ovn-kubernetes 26m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794 Created pod: payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw 26m Normal Started pod/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw Container started 26m Normal Created pod/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw Container created 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw Container created 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.311s (1.311s including waiting). Image size: 166612641 bytes. 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw Container started 26m Normal Pulled pod/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.801s (1.801s including waiting). Image size: 166612641 bytes. 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-9a15f4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-9a15f4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning Unhealthy pod/payload-processing-e2e-inf-a-45e508-7b86876698-pjzq8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-45e508-5bcb6b795f-d42tg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Readiness probe failed: Get "http://10.134.0.199:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Warning ProbeError pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Readiness probe error: Get "http://10.134.0.199:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Normal AddedLabel gateway/e2e-ipp-a-c6f83c Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-c6f83c 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-c6f83c-openshift-default Scaled up replica set e2e-ipp-a-c6f83c-openshift-default-9b6c88589 from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-c6f83c-openshift-default-9b6c88589 Created pod: e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv 25m Normal AddedInterface pod/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv Add eth0 [10.133.1.36/23] from ovn-kubernetes 25m Normal Created pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Container created 25m Normal Pulled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Container started 25m Normal Killing pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-4m7zk Stopping container istio-proxy 25m Normal Killing pod/e2e-inf-a-45e508-openshift-default-585ccbd46f-z4cm8 Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c6f83c Scaled up replica set payload-pre-processing-e2e-ipp-a-c6f83c-958f44595 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c6f83c Scaled up replica set payload-processing-e2e-ipp-a-c6f83c-995d7bdc4 from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4 Error creating: pods "payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c6f83c: serviceaccount "payload-processing-e2e-ipp-a-c6f83c" not found 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-84304f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595 Created pod: payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-dtglw 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 Created pod: e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595 Error creating: pods "payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c6f83c: serviceaccount "payload-processing-e2e-ipp-a-c6f83c" not found 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-dtglw Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-dtglw Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-dtglw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-dtglw Add eth0 [10.133.1.38/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4 Created pod: payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-hvw9t 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-84304f-openshift-default Scaled up replica set e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 from 3 to 4 25m Normal Created pod/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-hvw9t Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-hvw9t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-hvw9t Add eth0 [10.134.0.212/23] from ovn-kubernetes 25m Warning Unhealthy pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Startup probe failed: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused 25m Normal Pulled pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg Container created 25m Normal Started pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg Container started 25m Normal Started pod/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-hvw9t Container started 25m Warning ProbeError pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Startup probe error: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused... 25m Normal AddedInterface pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg Add eth0 [10.133.1.39/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc Created pod: e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-1dabb8-openshift-default Scaled up replica set e2e-ipp-a-1dabb8-openshift-default-b6d9979bc from 0 to 1 25m Normal AddedLabel gateway/e2e-ipp-a-1dabb8 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-1dabb8 25m Normal AddedInterface pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt Add eth0 [10.133.1.40/23] from ovn-kubernetes 25m Warning ProbeError pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg Startup probe error: Get "http://10.133.1.39:15021/healthz/ready": dial tcp 10.133.1.39:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg Startup probe failed: Get "http://10.133.1.39:15021/healthz/ready": dial tcp 10.133.1.39:15021: connect: connection refused 25m Normal Killing pod/payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 Stopping container payload-pre-processing 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c Error creating: pods "payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-1dabb8: serviceaccount "payload-processing-e2e-ipp-a-1dabb8" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c Created pod: payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-hpr6v 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-1dabb8 Scaled up replica set payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-1dabb8 Scaled up replica set payload-processing-e2e-ipp-a-1dabb8-586496bcdd from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-1dabb8-586496bcdd Created pod: payload-processing-e2e-ipp-a-1dabb8-586496bcdd-hhlk9 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-1dabb8-586496bcdd Error creating: pods "payload-processing-e2e-ipp-a-1dabb8-586496bcdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-1dabb8: serviceaccount "payload-processing-e2e-ipp-a-1dabb8" not found 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-1dabb8-586496bcdd-hhlk9 Add eth0 [10.133.1.42/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-hpr6v Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-hpr6v Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-a-1dabb8-586496bcdd-hhlk9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-hpr6v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-hpr6v Add eth0 [10.133.1.41/23] from ovn-kubernetes 25m Normal Started pod/payload-processing-e2e-ipp-a-1dabb8-586496bcdd-hhlk9 Container started 25m Normal Created pod/payload-processing-e2e-ipp-a-1dabb8-586496bcdd-hhlk9 Container created 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-329f7b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-329f7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c6f83c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c6f83c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal Killing pod/payload-pre-processing-e2e-ipp-b-09fc89-6dcd698c75-csl2b Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-ipp-b-09fc89-695597ffd8-bntdb Stopping container payload-processing 25m Warning Unhealthy pod/payload-processing-e2e-inf-b-57d336-5f55d66b86-xqdhw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-57d336-6f75d45b47-bsrt8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Created pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Container created 25m Normal Pulled pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Container started 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-09fc89-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-09fc89-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-329f7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-329f7b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning ProbeError pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Startup probe error: Get "http://10.133.1.23:15021/healthz/ready": dial tcp 10.133.1.23:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Startup probe failed: Get "http://10.133.1.23:15021/healthz/ready": dial tcp 10.133.1.23:15021: connect: connection refused 24m Warning Unhealthy pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/e2e-inf-b-57d336-openshift-default-854545cc57-jgfjn Stopping container istio-proxy 24m Normal Killing pod/e2e-inf-b-57d336-openshift-default-854545cc57-fgc9l Stopping container istio-proxy 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-09fc89-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-09fc89-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447 Error creating: pods "payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-09fc89: serviceaccount "payload-processing-e2e-ipp-b-09fc89" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-09fc89 Scaled up replica set payload-processing-e2e-ipp-b-09fc89-847d958946 from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-09fc89 Scaled up replica set payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447 from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-09fc89-847d958946 Error creating: pods "payload-processing-e2e-ipp-b-09fc89-847d958946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-09fc89: serviceaccount "payload-processing-e2e-ipp-b-09fc89" not found 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-09fc89-847d958946-mcjkf Add eth0 [10.134.0.215/23] from ovn-kubernetes 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447 Created pod: payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-dr4ch 24m Normal Pulled pod/payload-processing-e2e-ipp-b-09fc89-847d958946-mcjkf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-09fc89-847d958946 Created pod: payload-processing-e2e-ipp-b-09fc89-847d958946-mcjkf 24m Normal Started pod/payload-processing-e2e-ipp-b-09fc89-847d958946-mcjkf Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-dr4ch Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-dr4ch Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-processing-e2e-ipp-b-09fc89-847d958946-mcjkf Container created 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-dr4ch Container started 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-dr4ch Add eth0 [10.134.0.216/23] from ovn-kubernetes 24m Warning Unhealthy pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Readiness probe failed: Get "http://10.133.1.30:15021/healthz/ready": read tcp 10.133.0.2:56854->10.133.1.30:15021: read: connection reset by peer 24m Normal Started pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Container started 24m Normal Pulled pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Container created 24m Warning ProbeError pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Readiness probe error: Get "http://10.133.1.30:15021/healthz/ready": read tcp 10.133.0.2:56854->10.133.1.30:15021: read: connection reset by peer... 24m Normal Started pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Container started 24m Normal Created pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Container created 24m Normal Pulled pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Warning Unhealthy pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Startup probe failed: Get "http://10.134.0.208:15021/healthz/ready": dial tcp 10.134.0.208:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Startup probe error: Get "http://10.134.0.208:15021/healthz/ready": dial tcp 10.134.0.208:15021: connect: connection refused... 24m Warning ProbeError pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Startup probe error: Get "http://10.133.1.30:15021/healthz/ready": dial tcp 10.133.1.30:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Startup probe failed: Get "http://10.133.1.30:15021/healthz/ready": dial tcp 10.133.1.30:15021: connect: connection refused 24m Normal Killing pod/payload-processing-e2e-ipp-a-c6f83c-995d7bdc4-hvw9t Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c6f83c-958f44595-dtglw Stopping container payload-pre-processing 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-e803b6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-bcb810-openshift-default-56f69574df Created pod: e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-bcb810-openshift-default Scaled up replica set e2e-ipp-b-bcb810-openshift-default-56f69574df from 0 to 1 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-e803b6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal AddedLabel gateway/e2e-ipp-b-bcb810 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-bcb810 24m Normal AddedInterface pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Add eth0 [10.133.1.44/23] from ovn-kubernetes 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-bcb810 Scaled up replica set payload-processing-e2e-ipp-b-bcb810-75ccffb45d from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-bcb810 Scaled up replica set payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-bcb810-75ccffb45d Created pod: payload-processing-e2e-ipp-b-bcb810-75ccffb45d-hjkxs 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc Error creating: pods "payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bcb810: serviceaccount "payload-processing-e2e-ipp-b-bcb810" not found 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-bcb810-75ccffb45d Error creating: pods "payload-processing-e2e-ipp-b-bcb810-75ccffb45d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bcb810: serviceaccount "payload-processing-e2e-ipp-b-bcb810" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc Created pod: payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-qg64m 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-qg64m Add eth0 [10.134.0.217/23] from ovn-kubernetes 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-qg64m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-bcb810-75ccffb45d-hjkxs Add eth0 [10.134.0.218/23] from ovn-kubernetes 23m Normal Pulled pod/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-qg64m Container created 23m Normal Pulled pod/payload-processing-e2e-ipp-b-bcb810-75ccffb45d-hjkxs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-processing-e2e-ipp-b-bcb810-75ccffb45d-hjkxs Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-qg64m Container started 23m Normal Started pod/payload-processing-e2e-ipp-b-bcb810-75ccffb45d-hjkxs Container started 23m Normal Started pod/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv Container started 23m Normal Created pod/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv Container created 23m Normal Killing pod/payload-processing-e2e-ipp-a-1dabb8-586496bcdd-hhlk9 Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-1dabb8-f97d4446c-hpr6v Stopping container payload-pre-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f7a5fe-5f858d98c-zkgfw Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-f7a5fe-84b8dd9794-jsxnw Stopping container payload-processing 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-2ae845-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-2ae845-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning ProbeError pod/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv Startup probe error: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv Startup probe failed: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-e803b6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-e803b6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Stopping container istio-proxy 23m Warning Unhealthy pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-ipp-a-f7a5fe-openshift-default-b997867f9-rbpbg Readiness probe error: HTTP probe failed with statuscode: 503... 23m Normal Pulled pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt Container created 23m Normal Started pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt Container started 23m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 23m Normal Killing pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Stopping container istio-proxy 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Deleted pod: maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk 23m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 5 to 4 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c6f83c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c6f83c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-5zwdk Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt Startup probe error: Get "http://10.133.1.40:15021/healthz/ready": dial tcp 10.133.1.40:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt Startup probe failed: Get "http://10.133.1.40:15021/healthz/ready": dial tcp 10.133.1.40:15021: connect: connection refused 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c6f83c Scaled up replica set payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c6f83c Scaled up replica set payload-processing-e2e-ipp-a-c6f83c-7bdcffb664 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95 Created pod: payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-zwjhm 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664 Error creating: pods "payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c6f83c: serviceaccount "payload-processing-e2e-ipp-a-c6f83c" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664 Created pod: payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-wcfms 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95 Error creating: pods "payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c6f83c: serviceaccount "payload-processing-e2e-ipp-a-c6f83c" not found 23m Normal Created pod/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-wcfms Container created 23m Normal Started pod/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-wcfms Container started 23m Normal Pulled pod/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-wcfms Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-wcfms Add eth0 [10.134.0.220/23] from ovn-kubernetes 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-zwjhm Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-zwjhm Container started 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-zwjhm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-zwjhm Add eth0 [10.134.0.221/23] from ovn-kubernetes 23m Normal Created pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 Container created 23m Normal Started pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 Container started 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-36b46d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-36b46d-openshift-default Scaled up replica set e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 from 1 to 2 23m Normal AddedInterface pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 Add eth0 [10.133.1.48/23] from ovn-kubernetes 23m Normal Pulled pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 Created pod: e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 Created pod: e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-329f7b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-329f7b-openshift-default Scaled up replica set e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 from 1 to 2 23m Normal AddedInterface pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj Add eth0 [10.133.1.49/23] from ovn-kubernetes 23m Normal Started pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj Container started 23m Normal Created pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj Container created 23m Normal Pulled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-1dabb8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-1dabb8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning Unhealthy pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 Startup probe failed: Get "http://10.133.1.48:15021/healthz/ready": dial tcp 10.133.1.48:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 Startup probe error: Get "http://10.133.1.48:15021/healthz/ready": dial tcp 10.133.1.48:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj Startup probe failed: Get "http://10.133.1.49:15021/healthz/ready": dial tcp 10.133.1.49:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj Startup probe error: Get "http://10.133.1.49:15021/healthz/ready": dial tcp 10.133.1.49:15021: connect: connection refused... 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9 Error creating: pods "payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-1dabb8: serviceaccount "payload-processing-e2e-ipp-a-1dabb8" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b Created pod: payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-fwpxz 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b Error creating: pods "payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-1dabb8: serviceaccount "payload-processing-e2e-ipp-a-1dabb8" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9 Created pod: payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-9xxxz 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-1dabb8 Scaled up replica set payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-1dabb8 Scaled up replica set payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9 from 0 to 1 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-9xxxz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-9xxxz Container started 23m Normal Started pod/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-fwpxz Container started 23m Normal Created pod/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-fwpxz Container created 23m Normal Pulled pod/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-fwpxz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-fwpxz Add eth0 [10.132.0.168/23] from ovn-kubernetes 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-9xxxz Add eth0 [10.134.0.223/23] from ovn-kubernetes 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-9xxxz Container created 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-1dabb8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-1dabb8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-09fc89-868c8bb447-dr4ch Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-09fc89-847d958946-mcjkf Stopping container payload-processing 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-2ae845-openshift-default Scaled up replica set e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf from 1 to 2 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf Created pod: e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-2ae845-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 22m Normal Pulled pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f Created pod: e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp 22m Normal ScalingReplicaSet deployment/e2e-ipp-b-0906f4-openshift-default Scaled up replica set e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f from 0 to 1 22m Normal AddedLabel gateway/e2e-ipp-b-0906f4 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-0906f4 22m Normal Started pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Container started 22m Normal AddedInterface pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Add eth0 [10.134.0.224/23] from ovn-kubernetes 22m Normal Created pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Container created 22m Normal AddedInterface pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Add eth0 [10.133.1.50/23] from ovn-kubernetes 22m Normal Killing pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Stopping container istio-proxy 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895 Error creating: pods "payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0906f4: serviceaccount "payload-processing-e2e-ipp-b-0906f4" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-0906f4-76b779f4dd Created pod: payload-processing-e2e-ipp-b-0906f4-76b779f4dd-xq6x4 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-0906f4-76b779f4dd Error creating: pods "payload-processing-e2e-ipp-b-0906f4-76b779f4dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0906f4: serviceaccount "payload-processing-e2e-ipp-b-0906f4" not found 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-0906f4 Scaled up replica set payload-processing-e2e-ipp-b-0906f4-76b779f4dd from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895 Created pod: payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-f7rhd 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-0906f4 Scaled up replica set payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895 from 0 to 1 22m Warning ProbeError pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-ipp-b-09fc89-openshift-default-869878cbf5-sgssp Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning Unhealthy pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Startup probe failed: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-f7rhd Add eth0 [10.133.1.52/23] from ovn-kubernetes 22m Warning ProbeError pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Startup probe error: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused... 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-f7rhd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Pulled pod/payload-processing-e2e-ipp-b-0906f4-76b779f4dd-xq6x4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-f7rhd Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-f7rhd Container started 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-0906f4-76b779f4dd-xq6x4 Add eth0 [10.134.0.225/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-ipp-b-0906f4-76b779f4dd-xq6x4 Container started 22m Normal Created pod/payload-processing-e2e-ipp-b-0906f4-76b779f4dd-xq6x4 Container created 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-c7b96a-6cb5fbc5f5-l7s7s Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-shared-a-w0-c7b96a-7dc6895976-54nxc Stopping container payload-processing 22m Normal AddedLabel gateway/e2e-ipp-a-14b0c3 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-14b0c3 22m Normal Pulled pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f Created pod: e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-14b0c3-openshift-default Scaled up replica set e2e-ipp-a-14b0c3-openshift-default-846bddfb8f from 0 to 1 22m Normal Created pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Container created 22m Normal Started pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Container started 22m Normal AddedInterface pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Add eth0 [10.133.1.53/23] from ovn-kubernetes 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7 Error creating: pods "payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-14b0c3: serviceaccount "payload-processing-e2e-ipp-a-14b0c3" not found 22m Warning ProbeError pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Startup probe error: Get "http://10.133.1.44:15021/healthz/ready": dial tcp 10.133.1.44:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Startup probe failed: Get "http://10.133.1.44:15021/healthz/ready": dial tcp 10.133.1.44:15021: connect: connection refused 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-14b0c3 Scaled up replica set payload-processing-e2e-ipp-a-14b0c3-59db99bc46 from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-14b0c3-59db99bc46 Error creating: pods "payload-processing-e2e-ipp-a-14b0c3-59db99bc46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-14b0c3: serviceaccount "payload-processing-e2e-ipp-a-14b0c3" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-14b0c3 Scaled up replica set payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7 from 0 to 1 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-zjwr8 Container started 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-zjwr8 Add eth0 [10.134.0.226/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7 Created pod: payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-zjwr8 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-zjwr8 Container created 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-14b0c3-59db99bc46 Created pod: payload-processing-e2e-ipp-a-14b0c3-59db99bc46-x2jcd 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-zjwr8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-14b0c3-59db99bc46-x2jcd Add eth0 [10.134.0.227/23] from ovn-kubernetes 22m Normal Pulled pod/payload-processing-e2e-ipp-a-14b0c3-59db99bc46-x2jcd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-a-14b0c3-59db99bc46-x2jcd Container created 22m Normal Started pod/payload-processing-e2e-ipp-a-14b0c3-59db99bc46-x2jcd Container started 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-bcb810-d4d57fdbc-qg64m Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-bcb810-75ccffb45d-hjkxs Stopping container payload-processing 22m Normal Killing pod/e2e-shared-a-w0-c7b96a-openshift-default-766c4f5c5f-h8v5x Stopping container istio-proxy 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-437981-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc Created pod: e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-437981-openshift-default Scaled up replica set e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc from 3 to 4 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-e803b6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 22m Normal Created pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv Container created 22m Normal Pulled pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e803b6-openshift-default Scaled up replica set e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c from 1 to 2 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-bcb810-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal AddedInterface pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv Add eth0 [10.133.1.55/23] from ovn-kubernetes 22m Normal Started pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv Container started 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c Created pod: e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-bcb810-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal Created pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk Container created 22m Normal Started pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk Container started 22m Normal Pulled pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk Add eth0 [10.133.1.56/23] from ovn-kubernetes 22m Warning ProbeError pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk Startup probe error: Get "http://10.133.1.56:15021/healthz/ready": dial tcp 10.133.1.56:15021: connect: connection refused... 22m Warning ProbeError pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv Startup probe error: Get "http://10.133.1.55:15021/healthz/ready": dial tcp 10.133.1.55:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk Startup probe failed: Get "http://10.133.1.56:15021/healthz/ready": dial tcp 10.133.1.56:15021: connect: connection refused 22m Warning Unhealthy pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv Startup probe failed: Get "http://10.133.1.55:15021/healthz/ready": dial tcp 10.133.1.55:15021: connect: connection refused 22m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-e526ec-dcdcdc5f9-g5fq9 Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-shared-b-w0-e526ec-59659cfd46-r84hx Stopping container payload-processing 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-bcb810-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-bcb810-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Normal Killing pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-shared-b-w0-e526ec-openshift-default-6ffbb88df-wlgwf Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal Killing pod/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 Stopping container payload-processing 21m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9a15f4-openshift-default Scaled up replica set e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc from 1 to 2 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-9a15f4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 21m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc Created pod: e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 21m Normal Created pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 Container created 21m Normal Pulled pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 Container started 21m Normal AddedInterface pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 Add eth0 [10.132.0.169/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 Created pod: e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-329f7b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-329f7b-openshift-default Scaled up replica set e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 from 2 to 3 21m Normal AddedInterface pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Add eth0 [10.132.0.170/23] from ovn-kubernetes 21m Normal Pulled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Container started 21m Normal Created pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Container created 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-bcb810-cf5bc884 Error creating: pods "payload-processing-e2e-ipp-b-bcb810-cf5bc884-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bcb810: serviceaccount "payload-processing-e2e-ipp-b-bcb810" not found 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-bcb810 Scaled up replica set payload-processing-e2e-ipp-b-bcb810-cf5bc884 from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487 Created pod: payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-m8k5n 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487 Error creating: pods "payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bcb810: serviceaccount "payload-processing-e2e-ipp-b-bcb810" not found 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-bcb810 Scaled up replica set payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487 from 0 to 1 21m Warning ProbeError pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 Startup probe error: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused... 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-bcb810-cf5bc884 Created pod: payload-processing-e2e-ipp-b-bcb810-cf5bc884-7dvv6 21m Warning Unhealthy pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 Startup probe failed: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused 21m Normal Created pod/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-m8k5n Container created 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-m8k5n Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-m8k5n Add eth0 [10.134.0.229/23] from ovn-kubernetes 21m Normal Started pod/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-m8k5n Container started 21m Normal Started pod/payload-processing-e2e-ipp-b-bcb810-cf5bc884-7dvv6 Container started 21m Normal AddedInterface pod/payload-processing-e2e-ipp-b-bcb810-cf5bc884-7dvv6 Add eth0 [10.132.0.172/23] from ovn-kubernetes 21m Normal Pulled pod/payload-processing-e2e-ipp-b-bcb810-cf5bc884-7dvv6 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/payload-processing-e2e-ipp-b-bcb810-cf5bc884-7dvv6 Container created 21m Warning ProbeError pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Startup probe error: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Startup probe failed: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused 21m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-40cf9a-5ccdc94fff-2wxnp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-worker-w0-40cf9a-579fb97c7b-krvh6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e803b6-openshift-default Scaled up replica set e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c from 2 to 3 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c Created pod: e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-e803b6-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 21m Normal Pulled pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Add eth0 [10.132.0.182/23] from ovn-kubernetes 21m Normal Created pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Container created 21m Normal Started pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Container started 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-8f1627-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 21m Normal Started pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 Container started 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f Created pod: e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-8f1627-openshift-default Scaled up replica set e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f from 1 to 2 21m Normal AddedInterface pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 Add eth0 [10.133.1.79/23] from ovn-kubernetes 21m Normal Pulled pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 Container created 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-0906f4-6df6bdc895-f7rhd Stopping container payload-pre-processing 21m Warning ProbeError pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Startup probe error: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused... 21m Normal Killing pod/payload-processing-e2e-ipp-b-0906f4-76b779f4dd-xq6x4 Stopping container payload-processing 21m Warning Unhealthy pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Startup probe failed: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused 21m Warning ProbeError pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 Startup probe error: Get "http://10.133.1.79:15021/healthz/ready": dial tcp 10.133.1.79:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 Startup probe failed: Get "http://10.133.1.79:15021/healthz/ready": dial tcp 10.133.1.79:15021: connect: connection refused 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c6f83c-6f8d567c95-zwjhm Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-a-c6f83c-7bdcffb664-wcfms Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-14b0c3-6568595bb7-zjwr8 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-a-14b0c3-59db99bc46-x2jcd Stopping container payload-processing 20m Normal ScalingReplicaSet deployment/e2e-ipp-b-2d286f-openshift-default Scaled up replica set e2e-ipp-b-2d286f-openshift-default-6c84c45c89 from 0 to 1 20m Normal AddedLabel gateway/e2e-ipp-b-2d286f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-2d286f 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-2d286f-openshift-default-6c84c45c89 Created pod: e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l 20m Normal AddedInterface pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Add eth0 [10.134.0.233/23] from ovn-kubernetes 20m Normal Killing pod/e2e-ipp-a-c6f83c-openshift-default-9b6c88589-nx7fv Stopping container istio-proxy 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5 Created pod: payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-4xkd6 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5 Error creating: pods "payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-2d286f: serviceaccount "payload-processing-e2e-ipp-b-2d286f" not found 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8 Created pod: payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-dppph 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8 Error creating: pods "payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-2d286f: serviceaccount "payload-processing-e2e-ipp-b-2d286f" not found 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-2d286f Scaled up replica set payload-processing-e2e-ipp-b-2d286f-74d89cb4c8 from 0 to 1 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-2d286f Scaled up replica set payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5 from 0 to 1 20m Normal Pulled pod/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-dppph Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-dppph Container created 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-4xkd6 Add eth0 [10.134.0.234/23] from ovn-kubernetes 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-4xkd6 Container started 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-dppph Add eth0 [10.134.0.235/23] from ovn-kubernetes 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-4xkd6 Container created 20m Normal Started pod/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-dppph Container started 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-4xkd6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedLabel gateway/e2e-ipp-a-9ee1fc Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-9ee1fc 20m Normal ScalingReplicaSet deployment/e2e-ipp-a-9ee1fc-openshift-default Scaled up replica set e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5 from 0 to 1 20m Normal SuccessfulCreate replicaset/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5 Created pod: e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw 20m Normal AddedInterface pod/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw Add eth0 [10.134.0.236/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87 Created pod: payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-mjvbc 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-9ee1fc-5746854446 Error creating: pods "payload-processing-e2e-ipp-a-9ee1fc-5746854446-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-9ee1fc: serviceaccount "payload-processing-e2e-ipp-a-9ee1fc" not found 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-9ee1fc Scaled up replica set payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87 from 0 to 1 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-9ee1fc Scaled up replica set payload-processing-e2e-ipp-a-9ee1fc-5746854446 from 0 to 1 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87 Error creating: pods "payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-9ee1fc: serviceaccount "payload-processing-e2e-ipp-a-9ee1fc" not found 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-9ee1fc-5746854446 Created pod: payload-processing-e2e-ipp-a-9ee1fc-5746854446-rqk4w 20m Normal Started pod/payload-processing-e2e-ipp-a-9ee1fc-5746854446-rqk4w Container started 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-mjvbc Add eth0 [10.133.1.82/23] from ovn-kubernetes 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-mjvbc Container started 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-mjvbc Container created 20m Normal Created pod/payload-processing-e2e-ipp-a-9ee1fc-5746854446-rqk4w Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-a-9ee1fc-5746854446-rqk4w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-9ee1fc-5746854446-rqk4w Add eth0 [10.133.1.81/23] from ovn-kubernetes 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-mjvbc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal ScalingReplicaSet deployment/e2e-worker-w0-40cf9a-openshift-default (combined from similar events): Scaled up replica set e2e-worker-w0-40cf9a-openshift-default-7449b86f87 from 2 to 3 20m Normal SuccessfulCreate replicaset/e2e-worker-w0-40cf9a-openshift-default-7449b86f87 Created pod: e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-40cf9a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 20m Normal Started pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Container started 20m Normal Created pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Container created 20m Normal Pulled pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedLabel gateway/e2e-ipp-a-3e2b2c Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-3e2b2c 20m Normal AddedInterface pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m Add eth0 [10.134.0.237/23] from ovn-kubernetes 20m Normal Pulled pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal SuccessfulCreate replicaset/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65 Created pod: e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn 20m Normal Created pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m Container created 20m Normal AddedInterface pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Add eth0 [10.132.0.184/23] from ovn-kubernetes 20m Normal ScalingReplicaSet deployment/e2e-ipp-a-3e2b2c-openshift-default Scaled up replica set e2e-ipp-a-3e2b2c-openshift-default-54764b7f65 from 0 to 1 20m Normal Started pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m Container started 20m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c Created pod: e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-e803b6-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e803b6-openshift-default Scaled up replica set e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c from 3 to 4 20m Normal AddedInterface pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 Add eth0 [10.132.0.185/23] from ovn-kubernetes 20m Normal Started pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 Container started 20m Normal Pulled pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Created pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 Container created 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-8f1627-openshift-default Scaled up replica set e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f from 2 to 3 20m Normal Pulled pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 Add eth0 [10.132.0.186/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f Created pod: e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 20m Warning ProbeError pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Startup probe error: Get "http://10.133.1.53:15021/healthz/ready": dial tcp 10.133.1.53:15021: connect: connection refused... 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-8f1627-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 20m Warning Unhealthy pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Startup probe failed: Get "http://10.133.1.53:15021/healthz/ready": dial tcp 10.133.1.53:15021: connect: connection refused 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f Error creating: pods "payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3e2b2c: serviceaccount "payload-processing-e2e-ipp-a-3e2b2c" not found 20m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-lkrpw 20m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 20m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-3e2b2c Scaled up replica set payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f from 0 to 1 20m Normal Created pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 Container created 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f Created pod: payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-f54wd 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-3e2b2c Scaled up replica set payload-processing-e2e-ipp-a-3e2b2c-799768d694 from 0 to 1 20m Normal Started pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 Container started 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-3e2b2c-799768d694 Created pod: payload-processing-e2e-ipp-a-3e2b2c-799768d694-x5lm7 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-3e2b2c-799768d694 Error creating: pods "payload-processing-e2e-ipp-a-3e2b2c-799768d694-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3e2b2c: serviceaccount "payload-processing-e2e-ipp-a-3e2b2c" not found 20m Normal Created pod/payload-processing-e2e-ipp-a-3e2b2c-799768d694-x5lm7 Container created 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-f54wd Container created 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-f54wd Container started 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-f54wd Add eth0 [10.133.1.84/23] from ovn-kubernetes 20m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-lkrpw Add eth0 [10.132.0.187/23] from ovn-kubernetes 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-f54wd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Pulled pod/payload-processing-e2e-ipp-a-3e2b2c-799768d694-x5lm7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-processing-e2e-ipp-a-3e2b2c-799768d694-x5lm7 Container started 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-3e2b2c-799768d694-x5lm7 Add eth0 [10.133.1.85/23] from ovn-kubernetes 20m Warning ProbeError pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m Startup probe error: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused... 20m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-lkrpw Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 20m Warning Unhealthy pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m Startup probe failed: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused 20m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-lkrpw Container created 20m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-lkrpw Container started 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5 Error creating: pods "payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0906f4: serviceaccount "payload-processing-e2e-ipp-b-0906f4" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5 Created pod: payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-rf9mm 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-0906f4 Scaled up replica set payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5 from 0 to 1 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-0906f4-7d7f495d74 Error creating: pods "payload-processing-e2e-ipp-b-0906f4-7d7f495d74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0906f4: serviceaccount "payload-processing-e2e-ipp-b-0906f4" not found 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-0906f4 Scaled up replica set payload-processing-e2e-ipp-b-0906f4-7d7f495d74 from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-0906f4-7d7f495d74 Created pod: payload-processing-e2e-ipp-b-0906f4-7d7f495d74-qf86f 20m Warning ProbeError pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-rf9mm Add eth0 [10.134.0.238/23] from ovn-kubernetes 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-rf9mm Container created 20m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-lkrpw Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal Pulled pod/payload-processing-e2e-ipp-b-0906f4-7d7f495d74-qf86f Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-0906f4-7d7f495d74-qf86f Add eth0 [10.134.0.239/23] from ovn-kubernetes 20m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-lkrpw Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-rf9mm Container started 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-rf9mm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Warning Unhealthy pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 20m Normal Killing pod/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n Stopping container payload-pre-processing 20m Normal Created pod/payload-processing-e2e-ipp-b-0906f4-7d7f495d74-qf86f Container created 20m Normal Started pod/payload-processing-e2e-ipp-b-0906f4-7d7f495d74-qf86f Container started 20m Warning ProbeError pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 Startup probe error: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 Startup probe failed: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-2ae845-openshift-default Scaled up replica set e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf from 2 to 3 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-2ae845-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 20m Normal Pulled pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Add eth0 [10.132.0.188/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf Created pod: e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f 20m Normal Started pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Container started 20m Normal Created pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Container created 20m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 Created pod: e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj 20m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 Created pod: e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-36b46d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 20m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-36b46d-openshift-default Scaled up replica set e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 from 2 to 4 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-14b0c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-14b0c3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Pulled pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Created pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Container created 20m Normal Pulled pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Container started 20m Warning Unhealthy pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Readiness probe failed: Get "http://10.133.1.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Normal AddedInterface pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Add eth0 [10.133.1.86/23] from ovn-kubernetes 20m Normal AddedInterface pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Add eth0 [10.132.0.189/23] from ovn-kubernetes 20m Normal Created pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Container created 20m Normal Started pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Container started 20m Warning ProbeError pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Readiness probe error: Get "http://10.133.1.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 20m Normal AddedInterface pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q Add eth0 [10.134.0.240/23] from ovn-kubernetes 20m Normal Pulled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-329f7b-openshift-default Scaled up replica set e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 from 3 to 5 20m Normal Created pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Container created 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-329f7b-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 20m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 Created pod: e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q 20m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87 Created pod: e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 20m Normal Pulled pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 Add eth0 [10.133.1.87/23] from ovn-kubernetes 20m Normal Created pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q Container created 20m Warning Unhealthy pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 20m Warning ProbeError pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 20m Normal Pulled pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 Container started 20m Normal Started pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q Container started 20m Normal Started pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Container started 20m Normal Created pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 Container created 20m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-nl5t4 Stopping container istio-proxy 20m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-4v55m Stopping container istio-proxy 20m Normal Killing pod/e2e-worker-w0-40cf9a-openshift-default-7449b86f87-tzxh6 Stopping container istio-proxy 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-14b0c3 Scaled up replica set payload-processing-e2e-ipp-a-14b0c3-64f964956d from 0 to 1 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-14b0c3 Scaled up replica set payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795 from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-14b0c3-64f964956d Created pod: payload-processing-e2e-ipp-a-14b0c3-64f964956d-d45gc 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-14b0c3-64f964956d Error creating: pods "payload-processing-e2e-ipp-a-14b0c3-64f964956d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-14b0c3: serviceaccount "payload-processing-e2e-ipp-a-14b0c3" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795 Created pod: payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-b96qm 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795 Error creating: pods "payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-14b0c3: serviceaccount "payload-processing-e2e-ipp-a-14b0c3" not found 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-b96qm Container started 20m Warning Unhealthy pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-b96qm Container created 20m Warning ProbeError pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-14b0c3-64f964956d-d45gc Add eth0 [10.134.0.241/23] from ovn-kubernetes 20m Normal Pulled pod/payload-processing-e2e-ipp-a-14b0c3-64f964956d-d45gc Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-b96qm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-b96qm Add eth0 [10.132.0.190/23] from ovn-kubernetes 20m Normal Created pod/payload-processing-e2e-ipp-a-14b0c3-64f964956d-d45gc Container created 20m Normal Started pod/payload-processing-e2e-ipp-a-14b0c3-64f964956d-d45gc Container started 20m Warning Unhealthy pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Startup probe failed: Get "http://10.133.1.86:15021/healthz/ready": dial tcp 10.133.1.86:15021: connect: connection refused 20m Warning ProbeError pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Startup probe error: Get "http://10.133.1.86:15021/healthz/ready": dial tcp 10.133.1.86:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 Startup probe failed: Get "http://10.133.1.87:15021/healthz/ready": dial tcp 10.133.1.87:15021: connect: connection refused 20m Warning ProbeError pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q Startup probe error: Get "http://10.134.0.240:15021/healthz/ready": dial tcp 10.134.0.240:15021: connect: connection refused... 20m Normal Killing pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-mrq9q Startup probe failed: Get "http://10.134.0.240:15021/healthz/ready": dial tcp 10.134.0.240:15021: connect: connection refused 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-329f7b-6ddfd56fcb-vdg4p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Stopping container istio-proxy 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-329f7b-65cff495ff-tg24n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning ProbeError pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 Startup probe error: Get "http://10.133.1.87:15021/healthz/ready": dial tcp 10.133.1.87:15021: connect: connection refused... 20m Normal Killing pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-wntrj Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Startup probe failed: Get "http://10.133.1.50:15021/healthz/ready": dial tcp 10.133.1.50:15021: connect: connection refused 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-bcb810-6c7bf88487-m8k5n Stopping container payload-pre-processing 20m Warning ProbeError pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Startup probe error: Get "http://10.133.1.50:15021/healthz/ready": dial tcp 10.133.1.50:15021: connect: connection refused... 20m Normal Killing pod/payload-processing-e2e-ipp-b-bcb810-cf5bc884-7dvv6 Stopping container payload-processing 20m Warning ProbeError pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-46pqx Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-pxhj6 Stopping container istio-proxy 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-84304f-openshift-default Scaled down replica set e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 from 4 to 3 20m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5 Deleted pod: e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-84304f-openshift-default New size: 3; reason: All metrics below target 20m Normal Killing pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-hbvcg Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-shared-a-w1-329f7b-openshift-default-568bb78f87-xpll4 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-14b0c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-14b0c3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Normal Killing pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Stopping container istio-proxy 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 Stopping container payload-processing 19m Warning Unhealthy pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-ipp-b-bcb810-openshift-default-56f69574df-l9lns Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 Deleted pod: e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-6bb06a-openshift-default New size: 2; reason: All metrics below target 19m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-6bb06a-openshift-default Scaled down replica set e2e-shared-b-w6-6bb06a-openshift-default-df9874c47 from 3 to 2 19m Normal Killing pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-msc5v Stopping container istio-proxy 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-0906f4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-0906f4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl Stopping container payload-processing 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-e803b6-75b549ddf5-wgrwn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-e803b6-5cd68547cb-5j587 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-bjtll Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w2-afdbe8-openshift-default-77c5cc4d6c-ctfj5 Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-afdbe8-6665895cf5-hmzxj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-afdbe8-78bf8756d4-h68rl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z Stopping container payload-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-2d286f-74d89cb4c8-dppph Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-2d286f-6ffff46bf5-4xkd6 Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-5rts9 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-vcbkk Stopping container istio-proxy 19m Warning ProbeError pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-jrg8b Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning Unhealthy pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-b-w1-e803b6-openshift-default-79f8f89c4c-wpqbs Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning ProbeError pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Readiness probe error: Get "http://10.134.0.233:15021/healthz/ready": read tcp 10.134.0.2:52806->10.134.0.233:15021: read: connection reset by peer... 19m Warning Unhealthy pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Readiness probe failed: Get "http://10.134.0.233:15021/healthz/ready": read tcp 10.134.0.2:52806->10.134.0.233:15021: read: connection reset by peer 19m Normal Started pod/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw Container started 19m Normal Pulled pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw Container created 19m Normal Created pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Container created 19m Normal Pulled pod/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Started pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Container started 19m Normal Killing pod/payload-processing-e2e-ipp-a-9ee1fc-5746854446-rqk4w Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-9ee1fc-67cb496b87-mjvbc Stopping container payload-pre-processing 19m Normal Killing pod/payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf Stopping container payload-processing 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-9ee1fc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-9ee1fc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning Unhealthy pod/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw Startup probe failed: Get "http://10.134.0.236:15021/healthz/ready": dial tcp 10.134.0.236:15021: connect: connection refused 19m Warning ProbeError pod/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw Startup probe error: Get "http://10.134.0.236:15021/healthz/ready": dial tcp 10.134.0.236:15021: connect: connection refused... 18m Warning ProbeError pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Startup probe error: Get "http://10.134.0.233:15021/healthz/ready": dial tcp 10.134.0.233:15021: connect: connection refused... 18m Warning Unhealthy pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Startup probe failed: Get "http://10.134.0.233:15021/healthz/ready": dial tcp 10.134.0.233:15021: connect: connection refused 18m Normal SuccessfulCreate replicaset/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc Created pod: e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-1dabb8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-84304f-6b8c5bd7b-6hb7z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-84304f-dc8b6ccff-gdkvb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal ScalingReplicaSet deployment/e2e-ipp-a-1dabb8-openshift-default Scaled up replica set e2e-ipp-a-1dabb8-openshift-default-b6d9979bc from 1 to 2 18m Normal AddedInterface pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 Add eth0 [10.134.0.244/23] from ovn-kubernetes 18m Normal Started pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 Container started 18m Normal Created pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 Container created 18m Normal Pulled pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-2d286f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-2d286f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Normal Killing pod/payload-processing-e2e-ipp-a-3e2b2c-799768d694-x5lm7 Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-3e2b2c-5856fcbb5f-f54wd Stopping container payload-pre-processing 18m Warning Unhealthy pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 Startup probe failed: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused 18m Warning ProbeError pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 Startup probe error: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused... 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-879596-5b795898bc-hgvxj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-worker-w1-879596-7f976c6496-prjrf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning ProbeError pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Readiness probe error: Get "http://10.132.0.184:15021/healthz/ready": read tcp 10.132.0.2:34110->10.132.0.184:15021: read: connection reset by peer... 18m Warning Unhealthy pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Readiness probe failed: Get "http://10.132.0.184:15021/healthz/ready": read tcp 10.132.0.2:34110->10.132.0.184:15021: read: connection reset by peer 18m Normal Killing pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Stopping container istio-proxy 18m Normal Started pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Container started 18m Normal Killing pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Stopping container istio-proxy 18m Normal Pulled pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Killing pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-k5dcf Stopping container istio-proxy 18m Normal Created pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Container created 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-2d286f Scaled up replica set payload-processing-e2e-ipp-b-2d286f-79f5f996b4 from 0 to 1 18m Warning Unhealthy pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-492rm Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-2d286f-79f5f996b4 Created pod: payload-processing-e2e-ipp-b-2d286f-79f5f996b4-lk6fc 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-2d286f-79f5f996b4 Error creating: pods "payload-processing-e2e-ipp-b-2d286f-79f5f996b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-2d286f: serviceaccount "payload-processing-e2e-ipp-b-2d286f" not found 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59 Error creating: pods "payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-2d286f: serviceaccount "payload-processing-e2e-ipp-b-2d286f" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59 Created pod: payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-8tzh7 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-2d286f Scaled up replica set payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59 from 0 to 1 18m Warning Unhealthy pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-shared-b-w2-84304f-openshift-default-6f866bd9c5-p529r Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-8tzh7 Add eth0 [10.134.0.245/23] from ovn-kubernetes 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-8tzh7 Container started 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-8tzh7 Container created 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-8tzh7 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-processing-e2e-ipp-b-2d286f-79f5f996b4-lk6fc Container started 18m Normal Created pod/payload-processing-e2e-ipp-b-2d286f-79f5f996b4-lk6fc Container created 18m Normal Pulled pod/payload-processing-e2e-ipp-b-2d286f-79f5f996b4-lk6fc Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-2d286f-79f5f996b4-lk6fc Add eth0 [10.134.0.246/23] from ovn-kubernetes 18m Warning Unhealthy pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Startup probe failed: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused 18m Warning ProbeError pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Startup probe error: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused... 18m Normal Killing pod/payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s Stopping container payload-processing 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-9ee1fc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-9ee1fc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Killing pod/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq Stopping container payload-pre-processing 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c Error creating: pods "payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-9ee1fc: serviceaccount "payload-processing-e2e-ipp-a-9ee1fc" not found 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-9ee1fc-54c499f956 Error creating: pods "payload-processing-e2e-ipp-a-9ee1fc-54c499f956-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-9ee1fc: serviceaccount "payload-processing-e2e-ipp-a-9ee1fc" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c Created pod: payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-vt789 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-9ee1fc Scaled up replica set payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-9ee1fc-54c499f956 Created pod: payload-processing-e2e-ipp-a-9ee1fc-54c499f956-hqp5l 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-9ee1fc Scaled up replica set payload-processing-e2e-ipp-a-9ee1fc-54c499f956 from 0 to 1 18m Normal Created pod/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-vt789 Container created 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-vt789 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-vt789 Container started 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-vt789 Add eth0 [10.134.0.247/23] from ovn-kubernetes 18m Normal Created pod/payload-processing-e2e-ipp-a-9ee1fc-54c499f956-hqp5l Container created 18m Normal Started pod/payload-processing-e2e-ipp-a-9ee1fc-54c499f956-hqp5l Container started 18m Normal Pulled pod/payload-processing-e2e-ipp-a-9ee1fc-54c499f956-hqp5l Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-processing-e2e-ipp-a-9ee1fc-54c499f956-hqp5l Add eth0 [10.134.0.248/23] from ovn-kubernetes 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-2d286f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-2d286f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-3e2b2c Scaled up replica set payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55 from 0 to 1 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b Error creating: pods "payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3e2b2c: serviceaccount "payload-processing-e2e-ipp-a-3e2b2c" not found 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-3e2b2c Scaled up replica set payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b from 0 to 1 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55 Error creating: pods "payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3e2b2c: serviceaccount "payload-processing-e2e-ipp-a-3e2b2c" not found 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-b4tfd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-b4tfd Add eth0 [10.134.0.249/23] from ovn-kubernetes 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b Created pod: payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b 18m Normal AddedInterface pod/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b Add eth0 [10.133.1.100/23] from ovn-kubernetes 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55 Created pod: payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-b4tfd 18m Normal Pulling pod/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 18m Normal Created pod/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-b4tfd Container created 18m Normal Started pod/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-b4tfd Container started 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-3e2b2c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Started pod/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b Container started 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-3e2b2c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Created pod/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b Container created 18m Normal Pulled pod/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.611s (1.611s including waiting). Image size: 166612641 bytes. 18m Normal SuccessfulCreate replicaset/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df Created pod: e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 18m Normal ScalingReplicaSet deployment/e2e-ipp-b-eddb03-openshift-default Scaled up replica set e2e-ipp-b-eddb03-openshift-default-5bcd44f6df from 0 to 1 18m Normal AddedLabel gateway/e2e-ipp-b-eddb03 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-eddb03 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-8d3632-74d48c769f-8ggxq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal AddedInterface pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Add eth0 [10.133.1.101/23] from ovn-kubernetes 18m Warning Unhealthy pod/payload-processing-e2e-worker-w2-8d3632-c779784c8-dk28s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6npbh Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w1-879596-openshift-default-6b7d57dcc5-6kjv7 Stopping container istio-proxy 18m Normal Killing pod/payload-processing-e2e-ipp-a-14b0c3-64f964956d-d45gc Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-14b0c3-78cd988795-b96qm Stopping container payload-pre-processing 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-eddb03 Scaled up replica set payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7 from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-eddb03-66bc479ffd Created pod: payload-processing-e2e-ipp-b-eddb03-66bc479ffd-9qht6 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-eddb03-66bc479ffd Error creating: pods "payload-processing-e2e-ipp-b-eddb03-66bc479ffd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-eddb03: serviceaccount "payload-processing-e2e-ipp-b-eddb03" not found 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-eddb03 Scaled up replica set payload-processing-e2e-ipp-b-eddb03-66bc479ffd from 0 to 1 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7 Error creating: pods "payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-eddb03: serviceaccount "payload-processing-e2e-ipp-b-eddb03" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7 Created pod: payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-n4spc 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-n4spc Container started 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-eddb03-66bc479ffd-9qht6 Add eth0 [10.133.1.104/23] from ovn-kubernetes 18m Normal Started pod/payload-processing-e2e-ipp-b-eddb03-66bc479ffd-9qht6 Container started 18m Normal Pulled pod/payload-processing-e2e-ipp-b-eddb03-66bc479ffd-9qht6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-n4spc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-n4spc Container created 18m Normal Created pod/payload-processing-e2e-ipp-b-eddb03-66bc479ffd-9qht6 Container created 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-n4spc Add eth0 [10.133.1.103/23] from ovn-kubernetes 18m Normal Killing pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Stopping container istio-proxy 17m Normal ScalingReplicaSet deployment/e2e-ipp-b-c74e55-openshift-default Scaled up replica set e2e-ipp-b-c74e55-openshift-default-958874d7 from 0 to 1 17m Normal AddedLabel gateway/e2e-ipp-b-c74e55 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-c74e55 17m Normal SuccessfulCreate replicaset/e2e-ipp-b-c74e55-openshift-default-958874d7 Created pod: e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn 17m Normal AddedInterface pod/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn Add eth0 [10.133.1.106/23] from ovn-kubernetes 17m Warning Unhealthy pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-ipp-a-14b0c3-openshift-default-846bddfb8f-7cr4s Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-c74e55 Scaled up replica set payload-pre-processing-e2e-ipp-b-c74e55-b45d997f from 0 to 1 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-c74e55 Scaled up replica set payload-processing-e2e-ipp-b-c74e55-7b8b55996d from 0 to 1 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-c74e55-7b8b55996d Created pod: payload-processing-e2e-ipp-b-c74e55-7b8b55996d-zt9mc 17m Normal Pulled pod/payload-processing-e2e-ipp-b-c74e55-7b8b55996d-zt9mc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-c74e55-7b8b55996d-zt9mc Add eth0 [10.134.0.250/23] from ovn-kubernetes 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-c74e55-7b8b55996d Error creating: pods "payload-processing-e2e-ipp-b-c74e55-7b8b55996d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c74e55: serviceaccount "payload-processing-e2e-ipp-b-c74e55" not found 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-vsvld Container created 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-vsvld Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-vsvld Add eth0 [10.133.1.107/23] from ovn-kubernetes 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-vsvld Container started 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f Error creating: pods "payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c74e55: serviceaccount "payload-processing-e2e-ipp-b-c74e55" not found 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f Created pod: payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-vsvld 17m Normal Started pod/payload-processing-e2e-ipp-b-c74e55-7b8b55996d-zt9mc Container started 17m Normal Created pod/payload-processing-e2e-ipp-b-c74e55-7b8b55996d-zt9mc Container created 17m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-zpmjs Stopping container istio-proxy 17m Normal Killing pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-worker-w2-8d3632-openshift-default-5ddf4457d9-5rpfg Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/payload-pre-processing-e2e-ipp-a-1dabb8-7f8cb6f4c9-9xxxz Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-2d286f-79f5f996b4-lk6fc Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-2d286f-5bb646bf59-8tzh7 Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-ipp-a-1dabb8-6897c5bb8b-fwpxz Stopping container payload-processing 17m Normal Killing pod/e2e-ipp-b-2d286f-openshift-default-6c84c45c89-lz28l Stopping container istio-proxy 17m Normal SuccessfulDelete replicaset/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc Deleted pod: e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv 17m Normal Killing pod/payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 Stopping container payload-pre-processing 17m Normal Killing pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-wsnwv Stopping container istio-proxy 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-437981-openshift-default New size: 3; reason: All metrics below target 17m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-437981-openshift-default Scaled down replica set e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc from 4 to 3 17m Normal Pulled pod/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Created pod/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn Container created 17m Normal Started pod/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn Container started 17m Normal Created pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Container created 17m Normal Started pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Container started 17m Normal Pulled pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-437981-766d48cb5f-767r2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Stopping container istio-proxy 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-437981-654b69f766-w7klv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/payload-processing-e2e-ipp-b-eddb03-66bc479ffd-9qht6 Stopping container payload-processing 17m Normal Killing pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-xlqcw Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Stopping container istio-proxy 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-eddb03-9ddb46db7-n4spc Stopping container payload-pre-processing 16m Warning ProbeError pod/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn Startup probe error: Get "http://10.133.1.106:15021/healthz/ready": dial tcp 10.133.1.106:15021: connect: connection refused... 16m Warning Unhealthy pod/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn Startup probe failed: Get "http://10.133.1.106:15021/healthz/ready": dial tcp 10.133.1.106:15021: connect: connection refused 16m Warning ProbeError pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Startup probe error: Get "http://10.133.1.101:15021/healthz/ready": dial tcp 10.133.1.101:15021: connect: connection refused... 16m Warning ProbeError pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-eddb03-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-eddb03-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning Unhealthy pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Startup probe failed: Get "http://10.133.1.101:15021/healthz/ready": dial tcp 10.133.1.101:15021: connect: connection refused 16m Warning Unhealthy pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-rjwzv Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Readiness probe error: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused... 16m Warning Unhealthy pod/e2e-shared-a-w5-437981-openshift-default-7b4d6f69cc-cmqlx Readiness probe failed: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused 16m Normal Killing pod/payload-processing-e2e-ipp-b-c74e55-7b8b55996d-zt9mc Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-c74e55-b45d997f-vsvld Stopping container payload-pre-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf Stopping container payload-processing 16m Normal Killing pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-f7vwt Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-a-1dabb8-openshift-default-b6d9979bc-4cfl7 Stopping container istio-proxy 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-c74e55-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-c74e55-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-eddb03 Scaled up replica set payload-processing-e2e-ipp-b-eddb03-598b66b8d8 from 0 to 1 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-eddb03-8576855547 Created pod: payload-pre-processing-e2e-ipp-b-eddb03-8576855547-rjdcx 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-eddb03 Scaled up replica set payload-pre-processing-e2e-ipp-b-eddb03-8576855547 from 0 to 1 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-eddb03-8576855547 Error creating: pods "payload-pre-processing-e2e-ipp-b-eddb03-8576855547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-eddb03: serviceaccount "payload-processing-e2e-ipp-b-eddb03" not found 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-eddb03-598b66b8d8 Error creating: pods "payload-processing-e2e-ipp-b-eddb03-598b66b8d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-eddb03: serviceaccount "payload-processing-e2e-ipp-b-eddb03" not found 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-eddb03-598b66b8d8 Created pod: payload-processing-e2e-ipp-b-eddb03-598b66b8d8-lc677 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-eddb03-8576855547-rjdcx Container started 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-eddb03-8576855547-rjdcx Container created 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-eddb03-8576855547-rjdcx Add eth0 [10.134.0.252/23] from ovn-kubernetes 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-eddb03-8576855547-rjdcx Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-eddb03-598b66b8d8-lc677 Add eth0 [10.134.0.253/23] from ovn-kubernetes 16m Normal Pulled pod/payload-processing-e2e-ipp-b-eddb03-598b66b8d8-lc677 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal Created pod/payload-processing-e2e-ipp-b-eddb03-598b66b8d8-lc677 Container created 16m Normal Started pod/payload-processing-e2e-ipp-b-eddb03-598b66b8d8-lc677 Container started 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99 Error creating: pods "payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c74e55: serviceaccount "payload-processing-e2e-ipp-b-c74e55" not found 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8 Error creating: pods "payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c74e55: serviceaccount "payload-processing-e2e-ipp-b-c74e55" not found 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-c74e55-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-c74e55 Scaled up replica set payload-processing-e2e-ipp-b-c74e55-59b8c6cc8 from 0 to 1 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-c74e55-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-c74e55 Scaled up replica set payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99 from 0 to 1 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99 Created pod: payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-c4sv5 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-c4sv5 Container created 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8 Created pod: payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-p2rgc 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-c4sv5 Container started 16m Normal Started pod/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-p2rgc Container started 16m Normal Created pod/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-p2rgc Container created 16m Normal Pulled pod/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-p2rgc Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-p2rgc Add eth0 [10.134.0.254/23] from ovn-kubernetes 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-c4sv5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-c4sv5 Add eth0 [10.133.1.112/23] from ovn-kubernetes 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-eddb03-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-eddb03-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-dbb507-7dfdd6d664-xtqpf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-dbb507-59876dbb59-jfcl2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-n8grp Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Stopping container istio-proxy 16m Warning ProbeError pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/e2e-shared-b-w5-dbb507-openshift-default-7bf544d8cf-l6nmw Readiness probe failed: HTTP probe failed with statuscode: 503 16m Normal Killing pod/payload-processing-e2e-ipp-b-0906f4-7d7f495d74-qf86f Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-0906f4-6c8b7d59c5-rf9mm Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w5-0d12c5-dc776f966-t282d Stopping container payload-processing 16m Normal Killing pod/e2e-ipp-b-0906f4-openshift-default-7b6bf8dd5f-4wncp Stopping container istio-proxy 16m Normal Killing pod/payload-pre-processing-e2e-worker-w5-0d12c5-d9d745775-mvczb Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-a-9ee1fc-54c499f956-hqp5l Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-a-9ee1fc-574466b74c-vt789 Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-a-3e2b2c-bb56cc65b-5fg8b Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-a-3e2b2c-6f556cbd55-b4tfd Stopping container payload-pre-processing 15m Normal Killing pod/e2e-ipp-a-9ee1fc-openshift-default-587b4fdbd5-94skw Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Stopping container istio-proxy 15m Warning ProbeError pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-ipp-a-3e2b2c-openshift-default-54764b7f65-m6nvn Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Stopping container istio-proxy 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-e81b56-78cd9675f9-6v6j6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-e81b56-598655d846-svf7m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/payload-processing-e2e-ipp-b-eddb03-598b66b8d8-lc677 Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-eddb03-8576855547-rjdcx Stopping container payload-pre-processing 15m Normal Killing pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-c4wd6 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Stopping container istio-proxy 15m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-fpfrc Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning Unhealthy pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning Unhealthy pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-worker-w5-0d12c5-openshift-default-7cf588cbc7-5hwzc Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning ProbeError pod/e2e-shared-a-w6-e81b56-openshift-default-58d77f7d69-cmxgv Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/payload-processing-e2e-ipp-b-c74e55-59b8c6cc8-p2rgc Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-c74e55-69b95b5c99-c4sv5 Stopping container payload-pre-processing 15m Normal Killing pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Stopping container istio-proxy 15m Warning ProbeError pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-ipp-b-eddb03-openshift-default-5bcd44f6df-mpmx9 Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 15m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 4 to 3 15m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Deleted pod: maas-default-gateway-openshift-default-6f77ddbcb7-45vtp 15m Normal Killing pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Stopping container istio-proxy 15m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-45vtp Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal Killing pod/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs Stopping container payload-pre-processing 15m Normal Killing pod/e2e-ipp-b-c74e55-openshift-default-958874d7-hx8mn Stopping container istio-proxy 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq Stopping container payload-processing 15m Normal Killing pod/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 Stopping container payload-pre-processing 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-6bb06a-69bfbb5cf-bbvcs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-pgpr9 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w4-9a15f4-openshift-default-6b75fcbdbc-n2cln Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-65hdp Stopping container istio-proxy 14m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-36b46d-openshift-default Scaled down replica set e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 from 4 to 3 14m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-36b46d-openshift-default New size: 3; reason: All metrics below target 14m Normal Killing pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 Stopping container istio-proxy 14m Normal SuccessfulDelete replicaset/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987 Deleted pod: e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-f7rq4 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-6bb06a-59ccd794fb-hxhxb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-9a15f4-7c7d4bdbc5-mfbxc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-9a15f4-779b664b68-sw7zq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning ProbeError pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Readiness probe error: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused... 14m Warning Unhealthy pod/e2e-shared-b-w6-6bb06a-openshift-default-df9874c47-smqnn Readiness probe failed: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused 14m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn Stopping container payload-processing 14m Normal Killing pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Stopping container istio-proxy 14m Normal Killing pod/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 Stopping container payload-processing 14m Warning ProbeError pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-hwrz6 Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning Unhealthy pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-rtqqj Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-a-w3-36b46d-openshift-default-d5d76d987-9kqf5 Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-36b46d-754ddff76c-kj887 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-36b46d-78c99bbd9b-fz2mj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Stopping container istio-proxy 14m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-lkrpw Stopping container discovery 14m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-lkrpw 14m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 Stopping container payload-pre-processing 14m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 14m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 14m Normal Killing pod/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz Stopping container payload-processing 14m Warning Unhealthy pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-kzw2f Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-ftbp9 Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-b-w4-2ae845-openshift-default-7bb6f6fdcf-p42lq Readiness probe error: HTTP probe failed with statuscode: 503... 14m Normal Killing pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-fwdn4 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-z48b5 Stopping container istio-proxy 14m Normal Killing pod/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 Stopping container payload-processing 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-2ae845-748796f66d-7m2rn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw Stopping container payload-pre-processing 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-2ae845-5879bb5c87-fm2xd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Stopping container istio-proxy 14m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Readiness probe error: HTTP probe failed with statuscode: 503... 14m Normal Killing pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Stopping container istio-proxy 14m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-79kff Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning Unhealthy pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-b-w3-8f1627-openshift-default-6dc95df59f-dncl5 Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-worker-w6-92fd86-openshift-default-5c74588f5-npjv2 Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-92fd86-5d6556c999-7vmpz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-worker-w6-92fd86-6b4699fcfb-tnwj8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx Stopping container payload-pre-processing 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-8f1627-6d5765975-hv4m7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-8f1627-669f7d8ff6-q4hnz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-4a0287-7c4d7dff9f-vf2cw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-worker-w4-4a0287-cc6d67c5d-m5kn9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-t8xfr Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w4-4a0287-openshift-default-6b4449b6b5-qhsrk Stopping container istio-proxy 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-95672c-5bcdd68f75-ss8xx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-worker-w3-95672c-7dcb6ff684-rrcdb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Stopping container istio-proxy 13m Normal Killing pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-ggbqc Stopping container istio-proxy 13m Warning Unhealthy pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Readiness probe failed: HTTP probe failed with statuscode: 503 13m Warning ProbeError pod/e2e-worker-w3-95672c-openshift-default-5774db9bc-m9bfl Readiness probe error: HTTP probe failed with statuscode: 503... 13m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-bb564 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Add eth0 [10.133.1.134/23] from ovn-kubernetes 13m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 3 to 6 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Add eth0 [10.132.0.193/23] from ovn-kubernetes 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-kthhw 13m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Container created 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Add eth0 [10.134.1.0/23] from ovn-kubernetes 13m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Container started 13m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Container started 13m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Container created 13m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Container started 13m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Container created 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Startup probe error: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Startup probe failed: Get "http://10.133.1.134:15021/healthz/ready": dial tcp 10.133.1.134:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Startup probe error: Get "http://10.134.1.0:15021/healthz/ready": dial tcp 10.134.1.0:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Startup probe failed: Get "http://10.134.1.0:15021/healthz/ready": dial tcp 10.134.1.0:15021: connect: connection refused 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Startup probe failed: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Startup probe error: Get "http://10.133.1.134:15021/healthz/ready": dial tcp 10.133.1.134:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Startup probe failed: Get "http://10.134.1.0:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Startup probe failed: Get "http://10.133.1.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Startup probe error: Get "http://10.134.1.0:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Startup probe error: Get "http://10.133.1.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Startup probe failed: Get "http://10.132.0.193:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Startup probe error: Get "http://10.132.0.193:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Add eth0 [10.134.1.2/23] from ovn-kubernetes 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Add eth0 [10.134.1.1/23] from ovn-kubernetes 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6f77ddbcb7 from 6 to 10 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Add eth0 [10.133.1.136/23] from ovn-kubernetes 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6f77ddbcb7 Created pod: maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Add eth0 [10.132.0.194/23] from ovn-kubernetes 13m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: All metrics below target 13m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Container created 13m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Container started 13m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Container created 13m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Container started 13m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Container started 13m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Started pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Container started 13m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Container created 13m Normal Pulled pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Container created 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Startup probe failed: Get "http://10.132.0.194:15021/healthz/ready": dial tcp 10.132.0.194:15021: connect: connection refused 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Startup probe failed: Get "http://10.134.1.2:15021/healthz/ready": dial tcp 10.134.1.2:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Startup probe error: Get "http://10.134.1.1:15021/healthz/ready": dial tcp 10.134.1.1:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Startup probe failed: Get "http://10.133.1.136:15021/healthz/ready": dial tcp 10.133.1.136:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Startup probe error: Get "http://10.134.1.2:15021/healthz/ready": dial tcp 10.134.1.2:15021: connect: connection refused... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Startup probe error: Get "http://10.132.0.194:15021/healthz/ready": dial tcp 10.132.0.194:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Startup probe failed: Get "http://10.134.1.1:15021/healthz/ready": dial tcp 10.134.1.1:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Startup probe error: Get "http://10.133.1.136:15021/healthz/ready": dial tcp 10.133.1.136:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Readiness probe failed: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-lzjlp Readiness probe error: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 11m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 11m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-xc4fv Stopping container discovery 11m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-xc4fv 9m4s Normal SuccessfulCreate replicaset/e2e-worker-single-main-7e3223-openshift-default-7788864c8f Created pod: e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws 9m4s Normal ScalingReplicaSet deployment/e2e-worker-single-main-7e3223-openshift-default Scaled up replica set e2e-worker-single-main-7e3223-openshift-default-7788864c8f from 0 to 1 9m4s Normal AddedLabel gateway/e2e-worker-single-main-7e3223 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-7e3223 9m3s Normal AddedInterface pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Add eth0 [10.133.1.149/23] from ovn-kubernetes 8m58s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-7e3223-7bdb89488d Created pod: payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t 8m58s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-7e3223-7bdb89488d Error creating: pods "payload-processing-e2e-worker-single-main-7e3223-7bdb89488d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-7e3223: serviceaccount "payload-processing-e2e-worker-single-main-7e3223" not found 8m58s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-7e3223 Scaled up replica set payload-processing-e2e-worker-single-main-7e3223-7bdb89488d from 0 to 1 8m58s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-7e3223 Scaled up replica set payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5c7cf from 0 to 1 8m58s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5c7cf Error creating: pods "payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5c7cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-7e3223: serviceaccount "payload-processing-e2e-worker-single-main-7e3223" not found 8m58s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5c7cf Created pod: payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 8m57s Normal Created pod/payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t Container created 8m57s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t Add eth0 [10.134.1.5/23] from ovn-kubernetes 8m57s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m57s Normal Started pod/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 Container started 8m57s Normal Started pod/payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t Container started 8m57s Normal Created pod/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 Container created 8m57s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 Add eth0 [10.134.1.4/23] from ovn-kubernetes 8m57s Normal Pulled pod/payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m46s Warning ProbeError pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Readiness probe error: Get "http://10.133.1.149:15021/healthz/ready": read tcp 10.133.0.2:57456->10.133.1.149:15021: read: connection reset by peer... 8m46s Warning Unhealthy pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Readiness probe failed: Get "http://10.133.1.149:15021/healthz/ready": read tcp 10.133.0.2:57456->10.133.1.149:15021: read: connection reset by peer 8m45s Normal Started pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Container started 8m45s Normal Pulled pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m45s Normal Created pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Container created 8m39s Warning ProbeError pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Startup probe error: Get "http://10.133.1.149:15021/healthz/ready": dial tcp 10.133.1.149:15021: connect: connection refused... 8m39s Warning Unhealthy pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Startup probe failed: Get "http://10.133.1.149:15021/healthz/ready": dial tcp 10.133.1.149:15021: connect: connection refused 8m33s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-7e3223-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m33s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-7e3223-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m18s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-7e3223-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m18s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-7e3223-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m3s Normal Killing pod/payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t Stopping container payload-processing 8m3s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 Stopping container payload-pre-processing 7m55s Normal Killing pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Stopping container istio-proxy 7m53s Warning Unhealthy pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Readiness probe failed: HTTP probe failed with statuscode: 503 7m53s Warning ProbeError pod/e2e-worker-single-main-7e3223-openshift-default-7788864c8fcv8ws Readiness probe error: HTTP probe failed with statuscode: 503... 7m46s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-7e3223-dbdf5nv4j7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m46s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-7e3223-7bdb8948859k9t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m39s Normal SuccessfulCreate replicaset/e2e-worker-single-main-2250ca-openshift-default-57885cfb5c Created pod: e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 5m39s Normal ScalingReplicaSet deployment/e2e-worker-single-main-2250ca-openshift-default Scaled up replica set e2e-worker-single-main-2250ca-openshift-default-57885cfb5c from 0 to 1 5m39s Normal AddedLabel gateway/e2e-worker-single-main-2250ca Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-2250ca 5m38s Normal AddedInterface pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Add eth0 [10.133.1.156/23] from ovn-kubernetes 5m34s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-2250ca-54857dfd6 Created pod: payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj 5m34s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-2250ca-54857dfd6 Error creating: pods "payload-processing-e2e-worker-single-main-2250ca-54857dfd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2250ca: serviceaccount "payload-processing-e2e-worker-single-main-2250ca" not found 5m34s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-2250ca Scaled up replica set payload-processing-e2e-worker-single-main-2250ca-54857dfd6 from 0 to 1 5m34s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd97d495 Created pod: payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 5m34s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-2250ca Scaled up replica set payload-pre-processing-e2e-worker-single-main-2250ca-6bbd97d495 from 0 to 1 5m34s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd97d495 Error creating: pods "payload-pre-processing-e2e-worker-single-main-2250ca-6bbd97d495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2250ca: serviceaccount "payload-processing-e2e-worker-single-main-2250ca" not found 5m33s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m33s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj Add eth0 [10.134.1.8/23] from ovn-kubernetes 5m33s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 Add eth0 [10.134.1.7/23] from ovn-kubernetes 5m33s Normal Pulled pod/payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m33s Normal Created pod/payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj Container created 5m33s Normal Started pod/payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj Container started 5m33s Normal Created pod/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 Container created 5m33s Normal Started pod/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 Container started 5m21s Normal Pulled pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m21s Normal Created pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Container created 5m21s Normal Started pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Container started 5m21s Warning Unhealthy pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Readiness probe failed: Get "http://10.133.1.156:15021/healthz/ready": dial tcp 10.133.1.156:15021: connect: connection refused 5m21s Warning ProbeError pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Readiness probe error: Get "http://10.133.1.156:15021/healthz/ready": dial tcp 10.133.1.156:15021: connect: connection refused... 5m16s Warning ProbeError pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Startup probe error: Get "http://10.133.1.156:15021/healthz/ready": dial tcp 10.133.1.156:15021: connect: connection refused... 5m16s Warning Unhealthy pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Startup probe failed: Get "http://10.133.1.156:15021/healthz/ready": dial tcp 10.133.1.156:15021: connect: connection refused 4m54s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-2250ca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m54s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-2250ca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m40s Normal Killing pod/payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj Stopping container payload-processing 4m40s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 Stopping container payload-pre-processing 4m28s Normal Killing pod/e2e-worker-single-main-2250ca-openshift-default-57885cfb5cct2f2 Stopping container istio-proxy 4m22s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-2250ca-6bbd98qzw5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m21s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-2250ca-54857dfd6zt4fj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 3m41s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Readiness probe error: Get "http://10.134.1.0:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m41s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-dmvpn Readiness probe failed: Get "http://10.134.1.0:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m28s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Readiness probe error: Get "http://10.132.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m28s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Readiness probe error: Get "http://10.134.1.1:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m28s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Readiness probe error: Get "http://10.134.1.2:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m28s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-kz7rk Readiness probe failed: Get "http://10.132.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m28s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Readiness probe error: Get "http://10.133.1.136:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m28s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-bsvzx Readiness probe failed: Get "http://10.133.1.136:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m28s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-s6jn9 Readiness probe failed: Get "http://10.134.1.2:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m28s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-tw8xj Readiness probe failed: Get "http://10.134.1.1:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m41s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Readiness probe error: Get "http://10.132.0.193:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m41s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Readiness probe error: Get "http://10.133.1.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m41s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-bb564 Readiness probe failed: Get "http://10.132.0.193:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m41s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-kthhw Readiness probe failed: Get "http://10.133.1.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m18s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Readiness probe error: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m18s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-vsbwn Readiness probe failed: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m17s Warning ProbeError pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m17s Warning Unhealthy pod/maas-default-gateway-openshift-default-6f77ddbcb7-g2j2j Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)