=== events in openshift-ingress (2026-10-30T01:40:39Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 48m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-dhlj6 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-dhlj6 to ip-10-0-130-245.ec2.internal 46m Normal Scheduled pod/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 Successfully assigned openshift-ingress/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 to ip-10-0-139-66.ec2.internal 42m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj Successfully assigned openshift-ingress/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj to ip-10-0-130-245.ec2.internal 42m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp Successfully assigned openshift-ingress/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp to ip-10-0-130-245.ec2.internal 44m Normal Scheduled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Successfully assigned openshift-ingress/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx to ip-10-0-130-245.ec2.internal 46m Normal Scheduled pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Successfully assigned openshift-ingress/e2e-ait-573837e3-openshift-default-7779f764c-crrqn to ip-10-0-139-66.ec2.internal 22m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf to ip-10-0-130-245.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-pzzpg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-pzzpg to ip-10-0-139-66.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-5tsn5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-5tsn5 to ip-10-0-132-84.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w to ip-10-0-139-66.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-bpmp7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-bpmp7 to ip-10-0-132-84.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls to ip-10-0-139-66.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-38b47d-db764875-k8wxg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-38b47d-db764875-k8wxg to ip-10-0-132-84.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-q7v6x Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-q7v6x to ip-10-0-139-66.ec2.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-w94vx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-w94vx to ip-10-0-139-66.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-tqk48 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-tqk48 to ip-10-0-139-66.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-6cba49-76d444555-7qh5w Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-6cba49-76d444555-7qh5w to ip-10-0-132-84.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-fgsx4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-fgsx4 to ip-10-0-132-84.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-9h869 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-9h869 to ip-10-0-132-84.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-wz49m Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-wz49m to ip-10-0-139-66.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-4tm2m Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-4tm2m to ip-10-0-139-66.ec2.internal 48m Normal Scheduled pod/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 Successfully assigned openshift-ingress/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 to ip-10-0-132-84.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 to ip-10-0-132-84.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-a4fe0143-d94c99447-4lmxh Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a4fe0143-d94c99447-4lmxh to ip-10-0-139-66.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-szq9v Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-szq9v to ip-10-0-139-66.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-5xchw Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-5xchw to ip-10-0-132-84.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-95pgv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-95pgv to ip-10-0-139-66.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 to ip-10-0-130-245.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m to ip-10-0-130-245.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk to ip-10-0-139-66.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-jqjkm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-jqjkm to ip-10-0-132-84.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r to ip-10-0-132-84.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb to ip-10-0-132-84.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc to ip-10-0-130-245.ec2.internal 45m Normal Scheduled pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Successfully assigned openshift-ingress/e2e-api-a-353403-openshift-default-5f567587b9-q56px to ip-10-0-130-245.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s to ip-10-0-139-66.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz to ip-10-0-130-245.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt to ip-10-0-132-84.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf to ip-10-0-130-245.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd to ip-10-0-130-245.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl to ip-10-0-130-245.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx to ip-10-0-139-66.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn to ip-10-0-132-84.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h to ip-10-0-130-245.ec2.internal 43m Normal Scheduled pod/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj Successfully assigned openshift-ingress/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj to ip-10-0-130-245.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-derive-74b07396-69664fc94c-qdt6t Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-74b07396-69664fc94c-qdt6t to ip-10-0-139-66.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-xfbc4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-xfbc4 to ip-10-0-130-245.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-jxhrq Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-jxhrq to ip-10-0-130-245.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-b-601aea-6587946c4c-gzxrq Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-601aea-6587946c4c-gzxrq to ip-10-0-130-245.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-j7xm7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-j7xm7 to ip-10-0-132-84.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-a-610065-66c456c5f5-rsst7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-610065-66c456c5f5-rsst7 to ip-10-0-132-84.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-a-353403-75df964455-rttsx Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-353403-75df964455-rttsx to ip-10-0-130-245.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-82kds Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-82kds to ip-10-0-130-245.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-ait-573837e3-6d645bbdff-dpfv2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-573837e3-6d645bbdff-dpfv2 to ip-10-0-139-66.ec2.internal 43m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 52m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm to ip-10-0-139-66.ec2.internal 43m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv Successfully assigned openshift-ingress/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv to ip-10-0-130-245.ec2.internal 44m Normal Scheduled pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Successfully assigned openshift-ingress/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr to ip-10-0-130-245.ec2.internal 43m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 55m Normal Scheduled pod/payload-pre-processing-7b98bb678c-8q7vm Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-8q7vm to ip-10-0-139-66.ec2.internal 42m Warning FailedScheduling pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w to ip-10-0-139-66.ec2.internal 42m Normal Scheduled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd Successfully assigned openshift-ingress/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd to ip-10-0-130-245.ec2.internal 48m Normal Scheduled pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Successfully assigned openshift-ingress/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 to ip-10-0-139-66.ec2.internal 36m Normal Scheduled pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Successfully assigned openshift-ingress/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t to ip-10-0-130-245.ec2.internal 45m Normal Scheduled pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n Successfully assigned openshift-ingress/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n to ip-10-0-130-245.ec2.internal 29m Normal Scheduled pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 Successfully assigned openshift-ingress/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 to ip-10-0-139-66.ec2.internal 24m Normal Scheduled pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q Successfully assigned openshift-ingress/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q to ip-10-0-130-245.ec2.internal 47m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 to ip-10-0-132-84.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq to ip-10-0-130-245.ec2.internal 45m Normal Scheduled pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Successfully assigned openshift-ingress/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb to ip-10-0-132-84.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 to ip-10-0-130-245.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v to ip-10-0-139-66.ec2.internal 33m Normal Scheduled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b Successfully assigned openshift-ingress/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b to ip-10-0-139-66.ec2.internal 45m Normal Scheduled pod/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd Successfully assigned openshift-ingress/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd to ip-10-0-132-84.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q Successfully assigned openshift-ingress/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q to ip-10-0-130-245.ec2.internal 33m Normal Scheduled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Successfully assigned openshift-ingress/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x to ip-10-0-130-245.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl to ip-10-0-139-66.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx to ip-10-0-130-245.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx to ip-10-0-132-84.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p to ip-10-0-130-245.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 to ip-10-0-130-245.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s to ip-10-0-130-245.ec2.internal 39m Normal Scheduled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Successfully assigned openshift-ingress/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt to ip-10-0-139-66.ec2.internal 21m Normal Scheduled pod/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d Successfully assigned openshift-ingress/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d to ip-10-0-139-66.ec2.internal 34m Warning FailedScheduling pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Warning FailedScheduling pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Normal Scheduled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Successfully assigned openshift-ingress/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz to ip-10-0-132-84.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-mt-d8090513-69dcffc55b-xgmzw Successfully assigned openshift-ingress/payload-processing-e2e-mt-d8090513-69dcffc55b-xgmzw to ip-10-0-132-84.ec2.internal 43m Normal Scheduled pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Successfully assigned openshift-ingress/e2e-api-b-601aea-openshift-default-8559b84459-xs799 to ip-10-0-130-245.ec2.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc to ip-10-0-132-84.ec2.internal 24m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 to ip-10-0-132-84.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 Successfully assigned openshift-ingress/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 to ip-10-0-132-84.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-d01638-5668596858-chl8d Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-d01638-5668596858-chl8d to ip-10-0-130-245.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs to ip-10-0-139-66.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-b22r6 Successfully assigned openshift-ingress/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-b22r6 to ip-10-0-139-66.ec2.internal 28m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs to ip-10-0-130-245.ec2.internal 29m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 to ip-10-0-130-245.ec2.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm to ip-10-0-130-245.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-j758b Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-j758b to ip-10-0-139-66.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj to ip-10-0-139-66.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 to ip-10-0-139-66.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m to ip-10-0-130-245.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj to ip-10-0-130-245.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj to ip-10-0-132-84.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 to ip-10-0-132-84.ec2.internal 43m Warning FailedScheduling pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 Successfully assigned openshift-ingress/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 to ip-10-0-139-66.ec2.internal 24m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg to ip-10-0-130-245.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-migrate-1e708b2f-55698bbdc8-5gmp5 Successfully assigned openshift-ingress/payload-processing-e2e-migrate-1e708b2f-55698bbdc8-5gmp5 to ip-10-0-139-66.ec2.internal 24m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb to ip-10-0-132-84.ec2.internal 21m Normal Scheduled pod/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp Successfully assigned openshift-ingress/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp to ip-10-0-130-245.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b to ip-10-0-139-66.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-zwdzp Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-zwdzp to ip-10-0-132-84.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-cppx7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-cppx7 to ip-10-0-139-66.ec2.internal 43m Warning FailedScheduling pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj Successfully assigned openshift-ingress/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj to ip-10-0-130-245.ec2.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 to ip-10-0-132-84.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-6cba49-9df45c854-hqw6c Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-6cba49-9df45c854-hqw6c to ip-10-0-139-66.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-wskgt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-wskgt to ip-10-0-132-84.ec2.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-3e9921-6555c58b98-k6f8c Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-3e9921-6555c58b98-k6f8c to ip-10-0-130-245.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-3e9921-5f86b9565b-d5wpq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-3e9921-5f86b9565b-d5wpq to ip-10-0-130-245.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-38b47d-65c9d667c6-snwt5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-38b47d-65c9d667c6-snwt5 to ip-10-0-132-84.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx to ip-10-0-132-84.ec2.internal 43m Warning FailedScheduling pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 Successfully assigned openshift-ingress/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 to ip-10-0-139-66.ec2.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 to ip-10-0-139-66.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f to ip-10-0-139-66.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps to ip-10-0-132-84.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l to ip-10-0-132-84.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h to ip-10-0-130-245.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 to ip-10-0-130-245.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 to ip-10-0-139-66.ec2.internal 44m Normal Scheduled pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Successfully assigned openshift-ingress/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw to ip-10-0-130-245.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr to ip-10-0-132-84.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c to ip-10-0-132-84.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-1a42e5-8b4998c78-p9lfr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1a42e5-8b4998c78-p9lfr to ip-10-0-130-245.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-qzn6j Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-qzn6j to ip-10-0-130-245.ec2.internal 18m Normal Scheduled pod/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 Successfully assigned openshift-ingress/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 to ip-10-0-139-66.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 to ip-10-0-139-66.ec2.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr to ip-10-0-130-245.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf to ip-10-0-132-84.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m to ip-10-0-130-245.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth to ip-10-0-132-84.ec2.internal 21m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn to ip-10-0-139-66.ec2.internal 23m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 27m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l to ip-10-0-130-245.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-75bdf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-75bdf to ip-10-0-139-66.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b to ip-10-0-130-245.ec2.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb to ip-10-0-139-66.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v to ip-10-0-130-245.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz to ip-10-0-139-66.ec2.internal 21m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn to ip-10-0-132-84.ec2.internal 23m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-fslvf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-fslvf to ip-10-0-132-84.ec2.internal 43m Normal Scheduled pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Successfully assigned openshift-ingress/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn to ip-10-0-130-245.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-a-5f99e7-58f6878784-x95g8 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-5f99e7-58f6878784-x95g8 to ip-10-0-139-66.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-gtknt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-gtknt to ip-10-0-132-84.ec2.internal 28m Normal Scheduled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b Successfully assigned openshift-ingress/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b to ip-10-0-132-84.ec2.internal 23m Normal Scheduled pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Successfully assigned openshift-ingress/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz to ip-10-0-132-84.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l to ip-10-0-130-245.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-481249-65677df64-8bpkm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-481249-65677df64-8bpkm to ip-10-0-132-84.ec2.internal 61m Normal Scheduled pod/kube-auth-proxy-64d5b5b7d7-t2dm4 Successfully assigned openshift-ingress/kube-auth-proxy-64d5b5b7d7-t2dm4 to ip-10-0-139-66.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj to ip-10-0-132-84.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc Successfully assigned openshift-ingress/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc to ip-10-0-139-66.ec2.internal 32m Normal Scheduled pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Successfully assigned openshift-ingress/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 to ip-10-0-130-245.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Successfully assigned openshift-ingress/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx to ip-10-0-139-66.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-b-f96ede-55bbbb7db8-956nk Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-f96ede-55bbbb7db8-956nk to ip-10-0-132-84.ec2.internal 61m Normal Scheduled pod/kube-auth-proxy-64d5b5b7d7-nttd5 Successfully assigned openshift-ingress/kube-auth-proxy-64d5b5b7d7-nttd5 to ip-10-0-130-245.ec2.internal 47m Normal Scheduled pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx Successfully assigned openshift-ingress/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx to ip-10-0-130-245.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Successfully assigned openshift-ingress/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h to ip-10-0-132-84.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr to ip-10-0-132-84.ec2.internal 48m Normal Scheduled pod/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm Successfully assigned openshift-ingress/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm to ip-10-0-132-84.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 to ip-10-0-130-245.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Successfully assigned openshift-ingress/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 to ip-10-0-139-66.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 to ip-10-0-130-245.ec2.internal 44m Normal Scheduled pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Successfully assigned openshift-ingress/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 to ip-10-0-130-245.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 to ip-10-0-130-245.ec2.internal 25m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-wvbtj Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wvbtj to ip-10-0-130-245.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-b-a07920-5657fc66dd-vk2lw Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-a07920-5657fc66dd-vk2lw to ip-10-0-130-245.ec2.internal 27m Normal Scheduled pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm Successfully assigned openshift-ingress/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm to ip-10-0-132-84.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-b-97ceef-684cfb864-gxprj Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-97ceef-684cfb864-gxprj to ip-10-0-130-245.ec2.internal 24m Normal Scheduled pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Successfully assigned openshift-ingress/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 to ip-10-0-132-84.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-r4tcs Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r4tcs to ip-10-0-139-66.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 to ip-10-0-130-245.ec2.internal 61m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-5f555b57d9gtglk Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9gtglk to ip-10-0-139-66.ec2.internal 42m Normal Scheduled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm Successfully assigned openshift-ingress/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm to ip-10-0-130-245.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n to ip-10-0-132-84.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx to ip-10-0-139-66.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff to ip-10-0-139-66.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn to ip-10-0-139-66.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h to ip-10-0-132-84.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft to ip-10-0-132-84.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-25x6n Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-25x6n to ip-10-0-130-245.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 to ip-10-0-130-245.ec2.internal 41m Normal Scheduled pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Successfully assigned openshift-ingress/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv to ip-10-0-139-66.ec2.internal 44m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q to ip-10-0-139-66.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq to ip-10-0-130-245.ec2.internal 61m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-cbtxb Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-cbtxb to ip-10-0-139-66.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 to ip-10-0-139-66.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t to ip-10-0-139-66.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb Successfully assigned openshift-ingress/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb to ip-10-0-130-245.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd to ip-10-0-139-66.ec2.internal 33m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh to ip-10-0-139-66.ec2.internal 48m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 to ip-10-0-139-66.ec2.internal 50m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz to ip-10-0-132-84.ec2.internal 48m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw to ip-10-0-132-84.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z to ip-10-0-130-245.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h to ip-10-0-139-66.ec2.internal 48m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 to ip-10-0-139-66.ec2.internal 37m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 to ip-10-0-130-245.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr to ip-10-0-132-84.ec2.internal 27m Normal Scheduled pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Successfully assigned openshift-ingress/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m to ip-10-0-130-245.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 to ip-10-0-132-84.ec2.internal 5m22s Normal Scheduled pod/payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 to ip-10-0-130-245.ec2.internal 33m Normal Scheduled pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Successfully assigned openshift-ingress/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj to ip-10-0-130-245.ec2.internal 8m56s Normal Scheduled pod/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt to ip-10-0-130-245.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z Successfully assigned openshift-ingress/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p to ip-10-0-130-245.ec2.internal 51m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c to ip-10-0-139-66.ec2.internal 48m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m to ip-10-0-132-84.ec2.internal 49m Normal Scheduled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Successfully assigned openshift-ingress/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq to ip-10-0-139-66.ec2.internal 39m Normal Scheduled pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Successfully assigned openshift-ingress/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 to ip-10-0-139-66.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm Successfully assigned openshift-ingress/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd to ip-10-0-139-66.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Successfully assigned openshift-ingress/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq to ip-10-0-139-66.ec2.internal 21m Normal Scheduled pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Successfully assigned openshift-ingress/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m to ip-10-0-132-84.ec2.internal 21m Warning FailedScheduling pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 51m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq to ip-10-0-132-84.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z to ip-10-0-139-66.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb to ip-10-0-139-66.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm to ip-10-0-132-84.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk to ip-10-0-139-66.ec2.internal 5m22s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 to ip-10-0-132-84.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l to ip-10-0-132-84.ec2.internal 8m56s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr to ip-10-0-130-245.ec2.internal 23m Warning FailedScheduling pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Successfully assigned openshift-ingress/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv to ip-10-0-139-66.ec2.internal 52m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj to ip-10-0-139-66.ec2.internal 51m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl to ip-10-0-130-245.ec2.internal 72m Warning FailedScheduling pod/router-default-696c77d7b7-f5pjl no nodes available to schedule pods 49m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s to ip-10-0-130-245.ec2.internal 66m Warning FailedScheduling pod/router-default-696c77d7b7-f5pjl no nodes available to schedule pods 66m Normal Scheduled pod/router-default-696c77d7b7-f5pjl Successfully assigned openshift-ingress/router-default-696c77d7b7-f5pjl to ip-10-0-139-66.ec2.internal 18m Normal Scheduled pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c Successfully assigned openshift-ingress/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c to ip-10-0-139-66.ec2.internal 47m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b to ip-10-0-130-245.ec2.internal 48m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn to ip-10-0-139-66.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h Successfully assigned openshift-ingress/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h to ip-10-0-132-84.ec2.internal 48m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt to ip-10-0-132-84.ec2.internal 54m Normal Scheduled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Successfully assigned openshift-ingress/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w to ip-10-0-130-245.ec2.internal 48m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb to ip-10-0-132-84.ec2.internal 41m Normal Scheduled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 Successfully assigned openshift-ingress/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 to ip-10-0-130-245.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr to ip-10-0-130-245.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 Successfully assigned openshift-ingress/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 to ip-10-0-130-245.ec2.internal 36m Normal Scheduled pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 Successfully assigned openshift-ingress/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb to ip-10-0-139-66.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 to ip-10-0-132-84.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 to ip-10-0-139-66.ec2.internal 26m Normal Scheduled pod/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 Successfully assigned openshift-ingress/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 to ip-10-0-130-245.ec2.internal 33m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 to ip-10-0-139-66.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f to ip-10-0-130-245.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn to ip-10-0-132-84.ec2.internal 33m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n to ip-10-0-139-66.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp to ip-10-0-130-245.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj to ip-10-0-132-84.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj Successfully assigned openshift-ingress/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj to ip-10-0-132-84.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-a-820c04-7b94544c4b-5sgbd Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-820c04-7b94544c4b-5sgbd to ip-10-0-130-245.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Successfully assigned openshift-ingress/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr to ip-10-0-139-66.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv to ip-10-0-132-84.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh to ip-10-0-130-245.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 Successfully assigned openshift-ingress/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 to ip-10-0-132-84.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 to ip-10-0-139-66.ec2.internal 22m Warning FailedScheduling pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Warning FailedScheduling pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Normal Scheduled pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Successfully assigned openshift-ingress/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm to ip-10-0-130-245.ec2.internal 21m Normal Scheduled pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss Successfully assigned openshift-ingress/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss to ip-10-0-130-245.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-a-49dfaf-b8746c684-mpl56 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-49dfaf-b8746c684-mpl56 to ip-10-0-139-66.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn to ip-10-0-130-245.ec2.internal 27m Normal Scheduled pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Successfully assigned openshift-ingress/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv to ip-10-0-139-66.ec2.internal 53m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l to ip-10-0-139-66.ec2.internal 49m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 to ip-10-0-132-84.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 to ip-10-0-139-66.ec2.internal 49m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp to ip-10-0-130-245.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw to ip-10-0-139-66.ec2.internal 34m Warning FailedScheduling pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Warning FailedScheduling pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Normal Scheduled pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx Successfully assigned openshift-ingress/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx to ip-10-0-132-84.ec2.internal 49m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 to ip-10-0-139-66.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 Successfully assigned openshift-ingress/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 to ip-10-0-130-245.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-derive-74b07396-5bcdf47756-nvh8h Successfully assigned openshift-ingress/payload-processing-e2e-derive-74b07396-5bcdf47756-nvh8h to ip-10-0-139-66.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Successfully assigned openshift-ingress/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx to ip-10-0-132-84.ec2.internal 51m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp to ip-10-0-130-245.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-api-b-a32412-ccf56585d-ns54m Successfully assigned openshift-ingress/payload-processing-e2e-api-b-a32412-ccf56585d-ns54m to ip-10-0-130-245.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb Successfully assigned openshift-ingress/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb to ip-10-0-130-245.ec2.internal 29m Normal Scheduled pod/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 Successfully assigned openshift-ingress/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 to ip-10-0-130-245.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-api-b-73b32e-6655649548-fn9bv Successfully assigned openshift-ingress/payload-processing-e2e-api-b-73b32e-6655649548-fn9bv to ip-10-0-130-245.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 Successfully assigned openshift-ingress/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 to ip-10-0-130-245.ec2.internal 37m Normal Scheduled pod/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 Successfully assigned openshift-ingress/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 to ip-10-0-139-66.ec2.internal 46m Normal Scheduled pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Successfully assigned openshift-ingress/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v to ip-10-0-132-84.ec2.internal 26m Normal Scheduled pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc Successfully assigned openshift-ingress/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp to ip-10-0-130-245.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-api-b-601aea-68cd56fdcb-99xxg Successfully assigned openshift-ingress/payload-processing-e2e-api-b-601aea-68cd56fdcb-99xxg to ip-10-0-130-245.ec2.internal 34m Warning FailedScheduling pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Warning FailedScheduling pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Normal Scheduled pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq Successfully assigned openshift-ingress/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq to ip-10-0-130-245.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z Successfully assigned openshift-ingress/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z to ip-10-0-139-66.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-api-a-6867b8-757c78757b-7krfg Successfully assigned openshift-ingress/payload-processing-e2e-api-a-6867b8-757c78757b-7krfg to ip-10-0-130-245.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-api-a-610065-997f67946-54k2v Successfully assigned openshift-ingress/payload-processing-e2e-api-a-610065-997f67946-54k2v to ip-10-0-132-84.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-api-a-353403-55845f89db-zx8nt Successfully assigned openshift-ingress/payload-processing-e2e-api-a-353403-55845f89db-zx8nt to ip-10-0-130-245.ec2.internal 5m28s Normal Scheduled pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Successfully assigned openshift-ingress/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq to ip-10-0-130-245.ec2.internal 52m Normal Scheduled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Successfully assigned openshift-ingress/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st to ip-10-0-132-84.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-ait-c4e6c558-5f756664fb-gsb6t Successfully assigned openshift-ingress/payload-processing-e2e-ait-c4e6c558-5f756664fb-gsb6t to ip-10-0-130-245.ec2.internal 40m Normal Scheduled pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Successfully assigned openshift-ingress/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 to ip-10-0-130-245.ec2.internal 9m2s Normal Scheduled pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Successfully assigned openshift-ingress/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz to ip-10-0-139-66.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-ait-573837e3-65cfb8bddd-mqv4g Successfully assigned openshift-ingress/payload-processing-e2e-ait-573837e3-65cfb8bddd-mqv4g to ip-10-0-130-245.ec2.internal 38m Normal Scheduled pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr Successfully assigned openshift-ingress/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr to ip-10-0-139-66.ec2.internal 55m Normal Scheduled pod/payload-processing-5f78687d5b-rcxc5 Successfully assigned openshift-ingress/payload-processing-5f78687d5b-rcxc5 to ip-10-0-130-245.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh to ip-10-0-130-245.ec2.internal 17m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w to ip-10-0-139-66.ec2.internal 35m Normal Scheduled pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Successfully assigned openshift-ingress/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k to ip-10-0-132-84.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm to ip-10-0-139-66.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x to ip-10-0-132-84.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq to ip-10-0-139-66.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp to ip-10-0-132-84.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf to ip-10-0-139-66.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 to ip-10-0-139-66.ec2.internal 52m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r to ip-10-0-132-84.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c to ip-10-0-139-66.ec2.internal 52m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn to ip-10-0-139-66.ec2.internal 51m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 to ip-10-0-132-84.ec2.internal 51m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv to ip-10-0-130-245.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 to ip-10-0-130-245.ec2.internal 51m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p to ip-10-0-139-66.ec2.internal 39m Normal Scheduled pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Successfully assigned openshift-ingress/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c to ip-10-0-130-245.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt to ip-10-0-130-245.ec2.internal 47m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd to ip-10-0-132-84.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w to ip-10-0-139-66.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh to ip-10-0-132-84.ec2.internal 40m Normal Scheduled pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 Successfully assigned openshift-ingress/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 to ip-10-0-130-245.ec2.internal 47m Normal Scheduled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 Successfully assigned openshift-ingress/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h to ip-10-0-130-245.ec2.internal 54m Normal Scheduled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Successfully assigned openshift-ingress/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw to ip-10-0-130-245.ec2.internal 34m Normal Scheduled pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 Successfully assigned openshift-ingress/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 to ip-10-0-132-84.ec2.internal 34m Warning FailedScheduling pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 35m Warning FailedScheduling pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Normal Scheduled pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Successfully assigned openshift-ingress/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z to ip-10-0-139-66.ec2.internal 50m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv to ip-10-0-130-245.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd to ip-10-0-130-245.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 to ip-10-0-132-84.ec2.internal 30m Normal Scheduled pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv Successfully assigned openshift-ingress/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv to ip-10-0-132-84.ec2.internal 51m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p to ip-10-0-139-66.ec2.internal 51m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 to ip-10-0-132-84.ec2.internal 53m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr to ip-10-0-139-66.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 Successfully assigned openshift-ingress/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 to ip-10-0-132-84.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 to ip-10-0-139-66.ec2.internal 52m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm to ip-10-0-132-84.ec2.internal 48m Normal Scheduled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Successfully assigned openshift-ingress/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f to ip-10-0-130-245.ec2.internal 38m Normal Scheduled pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Successfully assigned openshift-ingress/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj to ip-10-0-139-66.ec2.internal 52m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs to ip-10-0-130-245.ec2.internal 30m Normal Scheduled pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms Successfully assigned openshift-ingress/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms to ip-10-0-139-66.ec2.internal 50m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b to ip-10-0-139-66.ec2.internal 54m Normal Scheduled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Successfully assigned openshift-ingress/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh to ip-10-0-130-245.ec2.internal 37m Normal Scheduled pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Successfully assigned openshift-ingress/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 to ip-10-0-130-245.ec2.internal 39m Normal Scheduled pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Successfully assigned openshift-ingress/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf to ip-10-0-130-245.ec2.internal 73m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-696c77d7b7 from 0 to 1 72m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 72m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 72m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 72m Normal SuccessfulCreate replicaset/router-default-696c77d7b7 Created pod: router-default-696c77d7b7-f5pjl 66m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 66m Warning FailedMount pod/router-default-696c77d7b7-f5pjl MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 66m Warning FailedMount pod/router-default-696c77d7b7-f5pjl MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Warning FailedMount pod/router-default-696c77d7b7-f5pjl MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Warning FailedMount pod/router-default-696c77d7b7-f5pjl MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 66m Normal Created pod/router-default-696c77d7b7-f5pjl Container created 66m Normal AddedInterface pod/router-default-696c77d7b7-f5pjl Add eth0 [10.132.0.19/23] from ovn-kubernetes 66m Normal Pulled pod/router-default-696c77d7b7-f5pjl Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 66m Normal Started pod/router-default-696c77d7b7-f5pjl Container started 61m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-64d5b5b7d7 from 0 to 2 61m Normal SuccessfulCreate replicaset/kube-auth-proxy-64d5b5b7d7 Created pod: kube-auth-proxy-64d5b5b7d7-nttd5 61m Normal SuccessfulCreate replicaset/kube-auth-proxy-64d5b5b7d7 Created pod: kube-auth-proxy-64d5b5b7d7-t2dm4 61m Normal AddedInterface pod/kube-auth-proxy-64d5b5b7d7-nttd5 Add eth0 [10.134.0.24/23] from ovn-kubernetes 61m Normal Pulling pod/kube-auth-proxy-64d5b5b7d7-nttd5 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 61m Normal AddedInterface pod/kube-auth-proxy-64d5b5b7d7-t2dm4 Add eth0 [10.132.0.31/23] from ovn-kubernetes 61m Normal Pulling pod/kube-auth-proxy-64d5b5b7d7-t2dm4 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 61m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-cbtxb 61m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 61m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-cbtxb Add eth0 [10.132.0.32/23] from ovn-kubernetes 61m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-cbtxb Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 61m Normal Pulled pod/kube-auth-proxy-64d5b5b7d7-nttd5 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.287s (2.287s including waiting). Image size: 188904677 bytes. 61m Normal Created pod/kube-auth-proxy-64d5b5b7d7-nttd5 Container created 61m Normal Started pod/kube-auth-proxy-64d5b5b7d7-nttd5 Container started 61m Normal Pulled pod/kube-auth-proxy-64d5b5b7d7-t2dm4 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.414s (2.414s including waiting). Image size: 188904677 bytes. 61m Normal Started pod/kube-auth-proxy-64d5b5b7d7-t2dm4 Container started 61m Normal Created pod/kube-auth-proxy-64d5b5b7d7-t2dm4 Container created 61m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-cbtxb Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.679s (2.679s including waiting). Image size: 185476477 bytes. 61m Normal Started pod/istiod-openshift-gateway-7876bfcb74-cbtxb Container started 61m Normal Created pod/istiod-openshift-gateway-7876bfcb74-cbtxb Container created 61m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-cbtxb Readiness probe failed: HTTP probe failed with statuscode: 503 61m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-cbtxb Readiness probe error: HTTP probe failed with statuscode: 503... 61m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-5f555b57d9 Created pod: data-science-gateway-data-science-gateway-class-5f555b57d9gtglk 61m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-5f555b57d9 from 0 to 1 61m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-5f555b57d9gtglk Add eth0 [10.132.0.33/23] from ovn-kubernetes 61m Normal Pulling pod/data-science-gateway-data-science-gateway-class-5f555b57d9gtglk Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 61m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Normal Pulled pod/data-science-gateway-data-science-gateway-class-5f555b57d9gtglk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.164s (9.164s including waiting). Image size: 183555333 bytes. 61m Normal Created pod/data-science-gateway-data-science-gateway-class-5f555b57d9gtglk Container created 61m Normal Started pod/data-science-gateway-data-science-gateway-class-5f555b57d9gtglk Container started 61m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-r4tcs 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 60m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-r4tcs Add eth0 [10.132.0.35/23] from ovn-kubernetes 60m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-r4tcs Container created 60m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-r4tcs Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 60m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-r4tcs Container started 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 60m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-cbtxb Stopping container discovery 60m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-cbtxb 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 0 to 1 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Add eth0 [10.132.0.45/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb 58m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 57m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Readiness probe error: Get "http://10.132.0.45:15021/healthz/ready": read tcp 10.132.0.2:37638->10.132.0.45:15021: read: connection reset by peer... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Readiness probe failed: Get "http://10.132.0.45:15021/healthz/ready": read tcp 10.132.0.2:37638->10.132.0.45:15021: read: connection reset by peer 57m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Container started 57m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Container created 57m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused... 56m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 55m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 55m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-rcxc5 55m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-8q7vm 55m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 55m Normal Pulling pod/payload-pre-processing-7b98bb678c-8q7vm Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 1 to 4 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 Add eth0 [10.133.0.31/23] from ovn-kubernetes 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Add eth0 [10.134.0.40/23] from ovn-kubernetes 55m Normal Pulling pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 55m Normal AddedInterface pod/payload-processing-5f78687d5b-rcxc5 Add eth0 [10.134.0.39/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 55m Normal Pulling pod/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 55m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Container started 55m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Container created 55m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-8q7vm Add eth0 [10.132.0.57/23] from ovn-kubernetes 55m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Add eth0 [10.132.0.58/23] from ovn-kubernetes 55m Normal Pulling pod/payload-processing-5f78687d5b-rcxc5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 55m Normal Pulled pod/payload-processing-5f78687d5b-rcxc5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.118s (2.118s including waiting). Image size: 125317101 bytes. 55m Normal Created pod/payload-processing-5f78687d5b-rcxc5 Container created 55m Normal Started pod/payload-processing-5f78687d5b-rcxc5 Container started 55m Normal Started pod/payload-pre-processing-7b98bb678c-8q7vm Container started 55m Normal Created pod/payload-pre-processing-7b98bb678c-8q7vm Container created 55m Normal Pulled pod/payload-pre-processing-7b98bb678c-8q7vm Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.573s (2.573s including waiting). Image size: 125317101 bytes. 55m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.456s (3.456s including waiting). Image size: 183555333 bytes. 55m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 Container started 55m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 Container created 55m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.583s (3.583s including waiting). Image size: 183555333 bytes. 55m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Container created 55m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Container started 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused... 55m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 Startup probe error: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-nv6m2 Startup probe failed: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 55m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Container created 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-7nffr 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Add eth0 [10.134.0.42/23] from ovn-kubernetes 55m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Container started 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 4 to 6 55m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: 55m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Add eth0 [10.133.0.32/23] from ovn-kubernetes 55m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Container created 55m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Container started 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-xsffc 55m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Startup probe error: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Startup probe failed: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 54m Normal AddedLabel gateway/e2e-worker-w1-beb8d5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-beb8d5 54m Normal AddedLabel gateway/e2e-worker-w2-40ac41 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-40ac41 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled up replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 0 to 1 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled up replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 0 to 1 54m Normal AddedLabel gateway/e2e-worker-w0-2b4105 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-2b4105 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p 54m Normal AddedLabel gateway/e2e-worker-w4-6add35 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-6add35 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled up replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 0 to 1 54m Normal AddedLabel gateway/e2e-worker-w6-5062f6 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-5062f6 54m Normal AddedLabel gateway/e2e-worker-w5-89a676 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-89a676 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp 54m Normal AddedLabel gateway/e2e-worker-w3-b3593a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-b3593a 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled up replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh 54m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Add eth0 [10.134.0.45/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled up replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh 54m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Add eth0 [10.134.0.43/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Add eth0 [10.134.0.46/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled up replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 0 to 1 54m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Add eth0 [10.134.0.44/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Add eth0 [10.134.0.48/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Add eth0 [10.134.0.47/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w 54m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Add eth0 [10.133.0.33/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled up replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-40ac41-776bf85c68 Error creating: pods "payload-processing-e2e-worker-w2-40ac41-776bf85c68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-40ac41: serviceaccount "payload-processing-e2e-worker-w2-40ac41" not found 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7 Created pod: payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7 Error creating: pods "payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-40ac41: serviceaccount "payload-processing-e2e-worker-w2-40ac41" not found 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-40ac41 Scaled up replica set payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7 from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-40ac41 Scaled up replica set payload-processing-e2e-worker-w2-40ac41-776bf85c68 from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-40ac41-776bf85c68 Created pod: payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq 54m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c Add eth0 [10.132.0.59/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq Container started 54m Normal Created pod/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c Container created 54m Normal AddedInterface pod/payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq Add eth0 [10.132.0.60/23] from ovn-kubernetes 54m Normal Started pod/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c Container started 54m Normal Created pod/payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq Container created 54m Normal Pulled pod/payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-6add35-7cd86547c6 Created pod: payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd 54m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-6add35-7cd86547c6 Error creating: pods "payload-processing-e2e-worker-w4-6add35-7cd86547c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-6add35: serviceaccount "payload-processing-e2e-worker-w4-6add35" not found 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999 Error creating: pods "payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-6add35: serviceaccount "payload-processing-e2e-worker-w4-6add35" not found 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999 Created pod: payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-6add35 Scaled up replica set payload-processing-e2e-worker-w4-6add35-7cd86547c6 from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-6add35 Scaled up replica set payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999 from 0 to 1 54m Normal Started pod/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 Add eth0 [10.132.0.61/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-beb8d5 Scaled up replica set payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695 from 0 to 1 54m Normal Started pod/payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd Container started 54m Normal Created pod/payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd Container created 54m Normal Pulled pod/payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695 Error creating: pods "payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-beb8d5: serviceaccount "payload-processing-e2e-worker-w1-beb8d5" not found 54m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb Error creating: pods "payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-beb8d5: serviceaccount "payload-processing-e2e-worker-w1-beb8d5" not found 54m Normal AddedInterface pod/payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd Add eth0 [10.132.0.62/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695 Created pod: payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-beb8d5 Scaled up replica set payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb from 0 to 1 54m Normal Created pod/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 Container created 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb Created pod: payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m 54m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr Add eth0 [10.132.0.65/23] from ovn-kubernetes 54m Normal Created pod/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr Container created 54m Normal AddedInterface pod/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m Add eth0 [10.132.0.64/23] from ovn-kubernetes 54m Normal Started pod/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr Container started 54m Normal Pulled pod/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m Container created 54m Normal Started pod/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m Container started 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c Created pod: payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm 54m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-5062f6-54896b9c4d Error creating: pods "payload-processing-e2e-worker-w6-5062f6-54896b9c4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-5062f6: serviceaccount "payload-processing-e2e-worker-w6-5062f6" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-5062f6-54896b9c4d Created pod: payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-5062f6 Scaled up replica set payload-processing-e2e-worker-w6-5062f6-54896b9c4d from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-5062f6 Scaled up replica set payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c Error creating: pods "payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-5062f6: serviceaccount "payload-processing-e2e-worker-w6-5062f6" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8 Created pod: payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 54m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8 Error creating: pods "payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-b3593a: serviceaccount "payload-processing-e2e-worker-w3-b3593a" not found 54m Normal Pulled pod/payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-b3593a Scaled up replica set payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9 from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9 Created pod: payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9 Error creating: pods "payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-b3593a: serviceaccount "payload-processing-e2e-worker-w3-b3593a" not found 54m Normal Started pod/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm Container started 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-b3593a Scaled up replica set payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8 from 0 to 1 54m Normal AddedInterface pod/payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq Add eth0 [10.132.0.67/23] from ovn-kubernetes 54m Normal Created pod/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm Container created 54m Normal Started pod/payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm Add eth0 [10.132.0.68/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq Container created 54m Normal AddedInterface pod/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 Add eth0 [10.132.0.69/23] from ovn-kubernetes 54m Normal Created pod/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 Container created 54m Normal Started pod/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb Container started 54m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb Add eth0 [10.132.0.70/23] from ovn-kubernetes 54m Normal Created pod/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb Container created 54m Normal Pulled pod/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 Container started 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-2b4105 Scaled up replica set payload-processing-e2e-worker-w0-2b4105-85f86769d6 from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-89a676 Scaled up replica set payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55 from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55 Error creating: pods "payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-89a676: serviceaccount "payload-processing-e2e-worker-w5-89a676" not found 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55 Created pod: payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h 54m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-89a676-5679c8db Error creating: pods "payload-processing-e2e-worker-w5-89a676-5679c8db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-89a676: serviceaccount "payload-processing-e2e-worker-w5-89a676" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-89a676-5679c8db Created pod: payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 54m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h Add eth0 [10.134.0.49/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-89a676 Scaled up replica set payload-processing-e2e-worker-w5-89a676-5679c8db from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-2b4105 Scaled up replica set payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47 from 0 to 1 54m Normal Pulled pod/payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr Add eth0 [10.134.0.50/23] from ovn-kubernetes 54m Normal Started pod/payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 Container started 54m Normal Started pod/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h Container started 54m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 Container created 54m Normal Pulled pod/payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr Container created 54m Normal AddedInterface pod/payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 Add eth0 [10.132.0.72/23] from ovn-kubernetes 54m Normal Started pod/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr Container started 54m Normal Created pod/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h Container created 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47 Created pod: payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47 Error creating: pods "payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2b4105: serviceaccount "payload-processing-e2e-worker-w0-2b4105" not found 54m Normal AddedInterface pod/payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt Add eth0 [10.134.0.51/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-2b4105-85f86769d6 Created pod: payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt 54m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-2b4105-85f86769d6 Error creating: pods "payload-processing-e2e-worker-w0-2b4105-85f86769d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2b4105: serviceaccount "payload-processing-e2e-worker-w0-2b4105" not found 54m Normal Created pod/payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt Container created 54m Normal Started pod/payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt Container started 54m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Container started 54m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Container created 54m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Readiness probe failed: Get "http://10.134.0.46:15021/healthz/ready": read tcp 10.134.0.2:54966->10.134.0.46:15021: read: connection reset by peer 54m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Readiness probe error: Get "http://10.134.0.46:15021/healthz/ready": read tcp 10.134.0.2:54966->10.134.0.46:15021: read: connection reset by peer... 54m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Container started 54m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Container created 54m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Readiness probe error: Get "http://10.134.0.45:15021/healthz/ready": read tcp 10.134.0.2:44684->10.134.0.45:15021: read: connection reset by peer... 54m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Readiness probe failed: Get "http://10.134.0.45:15021/healthz/ready": read tcp 10.134.0.2:44684->10.134.0.45:15021: read: connection reset by peer 54m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Readiness probe failed: Get "http://10.134.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Readiness probe error: Get "http://10.134.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Container started 54m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused 54m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Container created 53m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Container started 53m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Container created 53m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Container created 53m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Container started 53m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Container created 53m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Container started 53m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Startup probe error: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused... 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Container started 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Container created 53m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Startup probe failed: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Startup probe error: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused... 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Container created 53m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Container started 53m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Container created 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled up replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 1 to 2 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Add eth0 [10.132.0.76/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Add eth0 [10.132.0.77/23] from ovn-kubernetes 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Container started 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled up replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 1 to 2 53m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled up replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 1 to 2 52m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm Add eth0 [10.132.0.78/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm Container created 52m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm Container started 52m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 52m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Add eth0 [10.133.0.43/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled up replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 2 to 3 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st 52m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Container started 52m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Container created 52m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 52m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj 52m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs Container started 52m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs Container created 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs 52m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm Container started 52m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm Add eth0 [10.133.0.45/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm Container created 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled up replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 1 to 2 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled up replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 2 to 4 52m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs Add eth0 [10.134.0.63/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled up replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 1 to 3 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn 52m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Add eth0 [10.132.0.80/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn Add eth0 [10.132.0.79/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn Container created 52m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Container started 52m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r Container started 52m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r Container created 52m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r Add eth0 [10.133.0.46/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn Container started 52m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Container created 52m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh 51m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c Add eth0 [10.132.0.81/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c 51m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c Container created 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c Container started 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled up replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 1 to 2 51m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled up replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 1 to 2 51m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh Add eth0 [10.134.0.70/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh Container started 51m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh Container created 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h Add eth0 [10.134.0.71/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx Add eth0 [10.133.0.48/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 6 to 10 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf (combined from similar events): Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj 51m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h Container created 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj Add eth0 [10.134.0.72/23] from ovn-kubernetes 51m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h Container started 51m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx Container started 51m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj Container created 51m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj Container started 51m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f Container created 51m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx Container created 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f Add eth0 [10.132.0.82/23] from ovn-kubernetes 51m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f Container started 51m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled up replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 4 to 6 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p 51m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Add eth0 [10.133.0.51/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled up replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 3 to 6 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p 51m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Add eth0 [10.132.0.83/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled up replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 3 to 4 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 51m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Add eth0 [10.134.0.73/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Container created 51m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Container started 51m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Container created 51m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 Container started 51m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 Add eth0 [10.133.0.52/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 Container created 51m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Container started 51m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Container created 51m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p Add eth0 [10.132.0.84/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p Container created 51m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Container started 51m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p Container started 51m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Add eth0 [10.134.0.74/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Container started 51m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Container created 51m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled up replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 2 to 4 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Container started 51m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Add eth0 [10.134.0.79/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Container created 51m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Container started 51m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Add eth0 [10.133.0.54/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Container created 50m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled up replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 6 to 8 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled up replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 4 to 7 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw 50m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Container created 50m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Add eth0 [10.132.0.88/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Container created 50m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Add eth0 [10.132.0.87/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Container created 50m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Container started 50m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Container started 50m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Container created 50m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Container started 50m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Add eth0 [10.133.0.57/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Container started 50m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv Container started 50m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv Container created 50m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Add eth0 [10.132.0.89/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv Add eth0 [10.134.0.86/23] from ovn-kubernetes 50m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused... 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled up replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 6 to 10 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled up replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 2 to 3 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 (combined from similar events): Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled up replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 2 to 3 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Created pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz 50m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Add eth0 [10.133.0.59/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz Add eth0 [10.133.0.60/23] from ovn-kubernetes 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 50m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Container started 50m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Container created 50m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Add eth0 [10.134.0.88/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp Container started 50m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Container created 50m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Container created 50m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Container started 50m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Add eth0 [10.134.0.89/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz Container created 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq 50m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz Container started 50m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Container started 50m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp Add eth0 [10.133.0.61/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp Container created 50m Normal Started pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf Container started 50m Normal Created pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf Container created 50m Normal Pulled pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf Add eth0 [10.132.0.90/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq Add eth0 [10.132.0.91/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq Container created 50m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq Container started 50m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Readiness probe failed: Get "http://10.133.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled up replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 7 to 10 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff (combined from similar events): Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 (combined from similar events): Created pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd 49m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Readiness probe error: Get "http://10.133.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Created pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled up replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 8 to 10 49m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Container started 49m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Container started 49m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Add eth0 [10.134.0.97/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Container started 49m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Container created 49m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Container created 49m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Add eth0 [10.132.0.93/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Container started 49m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Add eth0 [10.133.0.62/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Add eth0 [10.133.0.63/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Container created 49m Normal Created pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Container created 49m Normal Pulled pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Add eth0 [10.134.0.96/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Container created 49m Normal Started pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Container started 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled up replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 3 to 4 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Add eth0 [10.134.0.101/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Container started 49m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Container created 49m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Readiness probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Readiness probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Readiness probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Readiness probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Readiness probe error: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Readiness probe error: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Readiness probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Readiness probe failed: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Readiness probe failed: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Readiness probe failed: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Readiness probe error: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Readiness probe failed: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Readiness probe error: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Readiness probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Readiness probe error: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Readiness probe failed: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Readiness probe error: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Readiness probe failed: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Readiness probe error: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Readiness probe failed: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Readiness probe error: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Readiness probe failed: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled up replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 3 to 5 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 49m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt Container started 49m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Add eth0 [10.132.0.96/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c 49m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt Container created 49m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s 49m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt Add eth0 [10.134.0.105/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Add eth0 [10.132.0.97/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Add eth0 [10.134.0.106/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Container started 49m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Container created 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled up replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 3 to 6 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled up replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 4 to 8 49m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Add eth0 [10.133.0.65/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l 49m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb 49m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Add eth0 [10.132.0.95/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w Container created 49m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w Container started 49m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Container created 49m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c Add eth0 [10.134.0.107/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Container started 49m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x Add eth0 [10.133.0.66/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Container started 49m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Container created 49m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x Container created 49m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c Container created 49m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w Add eth0 [10.132.0.98/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Container created 49m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Container started 49m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x Container started 49m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c Container started 49m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Container started 49m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Container created 49m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 (combined from similar events): Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z 49m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Add eth0 [10.133.0.69/23] from ovn-kubernetes 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 8; reason: All metrics below target 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled up replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 6 to 8 49m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Created pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 10; reason: 49m Normal AddedInterface pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Add eth0 [10.132.0.99/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled up replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 8 to 10 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt 49m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Container created 49m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Add eth0 [10.134.0.112/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Container started 49m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Add eth0 [10.132.0.100/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Container started 49m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Container created 49m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Container created 49m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Container created 49m Normal Started pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Container started 49m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Container started 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Readiness probe error: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Readiness probe error: Get "http://10.134.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Readiness probe failed: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Readiness probe error: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Readiness probe error: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Readiness probe error: Get "http://10.134.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Readiness probe failed: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Readiness probe failed: Get "http://10.134.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Readiness probe failed: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Readiness probe failed: Get "http://10.134.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Readiness probe failed: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Readiness probe error: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Readiness probe error: Get "http://10.132.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Readiness probe failed: Get "http://10.132.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled up replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 5 to 9 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb 48m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt Container created 48m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn Container created 48m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Add eth0 [10.134.0.117/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn Add eth0 [10.132.0.101/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb Container started 48m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Container created 48m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt Container started 48m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn Container started 48m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Container started 48m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb Container created 48m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt Add eth0 [10.133.0.71/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb Add eth0 [10.133.0.72/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-dhlj6 48m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 48m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Readiness probe failed: Get "http://10.134.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Readiness probe error: Get "http://10.134.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Readiness probe error: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Readiness probe failed: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/payload-processing-5f78687d5b-rcxc5 Readiness probe failed: timeout: failed to connect service "10.134.0.39:50052" within 1s: context deadline exceeded 48m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Readiness probe failed: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Readiness probe error: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-dhlj6 Add eth0 [10.134.0.119/23] from ovn-kubernetes 48m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-dhlj6 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 48m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 48m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-dhlj6 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.448s (3.448s including waiting). Image size: 185476477 bytes. 48m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 48m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-dhlj6 Container created 48m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 48m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-dhlj6 Container started 48m Normal SuccessfulCreate replicaset/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f Created pod: e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 48m Normal ScalingReplicaSet deployment/e2e-migrate-1e708b2f-gw-openshift-default Scaled up replica set e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f from 0 to 1 48m Normal AddedLabel gateway/e2e-migrate-1e708b2f-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-1e708b2f-gw 48m Normal AddedInterface pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Add eth0 [10.132.0.103/23] from ovn-kubernetes 48m Normal AddedLabel gateway/e2e-derive-74b07396 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-74b07396 48m Normal SuccessfulCreate replicaset/e2e-derive-74b07396-openshift-default-58f4bbdb7c Created pod: e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm 48m Normal Started pod/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm Container started 48m Normal Created pod/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm Container created 48m Normal Pulled pod/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm Add eth0 [10.133.0.75/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-derive-74b07396-openshift-default Scaled up replica set e2e-derive-74b07396-openshift-default-58f4bbdb7c from 0 to 1 48m Normal AddedLabel gateway/e2e-ait-c4e6c558 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c4e6c558 48m Warning ProbeError pod/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 48m Normal SuccessfulCreate replicaset/e2e-ait-c4e6c558-openshift-default-7977ff48f9 Created pod: e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 48m Normal ScalingReplicaSet deployment/e2e-ait-c4e6c558-openshift-default Scaled up replica set e2e-ait-c4e6c558-openshift-default-7977ff48f9 from 0 to 1 48m Warning Unhealthy pod/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 48m Normal Started pod/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 Container started 48m Warning FailedCreate replicaset/payload-processing-e2e-migrate-1e708b2f-55698bbdc8 Error creating: pods "payload-processing-e2e-migrate-1e708b2f-55698bbdc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-1e708b2f: serviceaccount "payload-processing-e2e-migrate-1e708b2f" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-1e708b2f-55698bbdc8 Created pod: payload-processing-e2e-migrate-1e708b2f-55698bbdc8-5gmp5 48m Normal AddedInterface pod/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 Add eth0 [10.133.0.76/23] from ovn-kubernetes 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7 Error creating: pods "payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-1e708b2f: serviceaccount "payload-processing-e2e-migrate-1e708b2f" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-1e708b2f Scaled up replica set payload-processing-e2e-migrate-1e708b2f-55698bbdc8 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-1e708b2f Scaled up replica set payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7 Created pod: payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-4tm2m 48m Normal Created pod/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 Container created 48m Normal Pulled pod/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-migrate-1e708b2f-55698bbdc8-5gmp5 Add eth0 [10.132.0.105/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-4tm2m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-4tm2m Add eth0 [10.132.0.104/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 48m Normal Created pod/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-4tm2m Container created 48m Normal Started pod/payload-processing-e2e-migrate-1e708b2f-55698bbdc8-5gmp5 Container started 48m Normal Created pod/payload-processing-e2e-migrate-1e708b2f-55698bbdc8-5gmp5 Container created 48m Normal Pulled pod/payload-processing-e2e-migrate-1e708b2f-55698bbdc8-5gmp5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-4tm2m Container started 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-74b07396 Scaled up replica set payload-processing-e2e-derive-74b07396-5bcdf47756 from 0 to 1 48m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Readiness probe failed: Get "http://10.132.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-74b07396 Scaled up replica set payload-pre-processing-e2e-derive-74b07396-69664fc94c from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-derive-74b07396-5bcdf47756 Error creating: pods "payload-processing-e2e-derive-74b07396-5bcdf47756-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-74b07396: serviceaccount "payload-processing-e2e-derive-74b07396" not found 48m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Readiness probe failed: Get "http://10.134.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Readiness probe error: Get "http://10.134.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-74b07396-5bcdf47756 Created pod: payload-processing-e2e-derive-74b07396-5bcdf47756-nvh8h 48m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Readiness probe error: Get "http://10.132.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-74b07396-69664fc94c Error creating: pods "payload-pre-processing-e2e-derive-74b07396-69664fc94c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-74b07396: serviceaccount "payload-processing-e2e-derive-74b07396" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-74b07396-69664fc94c Created pod: payload-pre-processing-e2e-derive-74b07396-69664fc94c-qdt6t 48m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Readiness probe error: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Readiness probe failed: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-c4e6c558 Scaled up replica set payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f from 0 to 1 48m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Readiness probe error: Get "http://10.134.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 48m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Readiness probe failed: Get "http://10.134.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-c4e6c558 Scaled up replica set payload-processing-e2e-ait-c4e6c558-5f756664fb from 0 to 1 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled up replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 4 to 8 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m 48m Warning FailedCreate replicaset/payload-processing-e2e-ait-c4e6c558-5f756664fb Error creating: pods "payload-processing-e2e-ait-c4e6c558-5f756664fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c4e6c558: serviceaccount "payload-processing-e2e-ait-c4e6c558" not found 48m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m Container started 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f Created pod: payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-82kds 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-c4e6c558-5f756664fb Created pod: payload-processing-e2e-ait-c4e6c558-5f756664fb-gsb6t 48m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m Add eth0 [10.133.0.79/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m Container created 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f Error creating: pods "payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c4e6c558: serviceaccount "payload-processing-e2e-ait-c4e6c558" not found 48m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw Container started 48m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw Container created 48m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw Add eth0 [10.133.0.78/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-ait-c4e6c558-5f756664fb-gsb6t Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-derive-74b07396-69664fc94c-qdt6t Add eth0 [10.132.0.106/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-derive-74b07396-69664fc94c-qdt6t Container started 48m Normal Created pod/payload-pre-processing-e2e-derive-74b07396-69664fc94c-qdt6t Container created 48m Normal AddedInterface pod/payload-processing-e2e-ait-c4e6c558-5f756664fb-gsb6t Add eth0 [10.134.0.123/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-derive-74b07396-69664fc94c-qdt6t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Pulled pod/payload-processing-e2e-ait-c4e6c558-5f756664fb-gsb6t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-ait-c4e6c558-5f756664fb-gsb6t Container created 48m Normal AddedInterface pod/payload-processing-e2e-derive-74b07396-5bcdf47756-nvh8h Add eth0 [10.132.0.107/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 Add eth0 [10.132.0.108/23] from ovn-kubernetes 48m Normal AddedInterface pod/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-82kds Add eth0 [10.134.0.122/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-82kds Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-82kds Container created 48m Normal Started pod/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-82kds Container started 48m Normal Pulled pod/payload-processing-e2e-derive-74b07396-5bcdf47756-nvh8h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-derive-74b07396-5bcdf47756-nvh8h Container created 48m Normal Started pod/payload-processing-e2e-derive-74b07396-5bcdf47756-nvh8h Container started 48m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 Add eth0 [10.132.0.109/23] from ovn-kubernetes 48m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 Container created 48m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 Container created 48m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 Container started 48m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 Container started 48m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused... 47m Normal SuccessfulCreate replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 (combined from similar events): Created pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled up replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 9 to 10 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled up replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 8 to 9 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 9; reason: 47m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z Container started 47m Normal Pulled pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b Container created 47m Normal Started pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b Container started 47m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z Container created 47m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z Add eth0 [10.134.0.128/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b Add eth0 [10.134.0.127/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-qbkz5 47m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 47m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Add eth0 [10.133.0.83/23] from ovn-kubernetes 47m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Container created 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c4e6c558-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Container started 47m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.574s (3.574s including waiting). Image size: 185476477 bytes. 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c4e6c558-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z Startup probe failed: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z Startup probe error: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 47m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled up replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 8 to 10 47m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f (combined from similar events): Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 47m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd Add eth0 [10.133.0.84/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7 Created pod: e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx 47m Normal AddedLabel gateway/e2e-shared-a-w6-ea2547 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-ea2547 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-ea2547-openshift-default Scaled up replica set e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7 from 0 to 1 47m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-74b07396-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-74b07396-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd Container started 47m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd Container created 47m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 Container created 47m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 Add eth0 [10.134.0.129/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 Container started 47m Normal AddedInterface pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx Add eth0 [10.134.0.130/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-ea2547 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-ea2547 Scaled up replica set payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8 from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-ea2547: serviceaccount "payload-processing-e2e-shared-a-w6-ea2547" not found 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8 Error creating: pods "payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-ea2547: serviceaccount "payload-processing-e2e-shared-a-w6-ea2547" not found 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 Add eth0 [10.132.0.111/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 Startup probe error: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 Startup probe failed: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8 Created pod: payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5 Created pod: payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 47m Normal Pulled pod/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b Container created 47m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b Add eth0 [10.132.0.112/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b Container started 47m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 Container created 47m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 Container started 47m Normal Started pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Container started 47m Normal Created pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Container created 47m Normal Pulled pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-1e708b2f-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-1e708b2f-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal Killing pod/payload-pre-processing-e2e-migrate-1e708b2f-666c96b4d7-4tm2m Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-migrate-1e708b2f-55698bbdc8-5gmp5 Stopping container payload-processing 47m Warning ProbeError pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 47m Normal Killing pod/payload-pre-processing-e2e-derive-74b07396-69664fc94c-qdt6t Stopping container payload-pre-processing 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled up replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 9 to 10 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff (combined from similar events): Created pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h 47m Normal Killing pod/payload-processing-e2e-derive-74b07396-5bcdf47756-nvh8h Stopping container payload-processing 47m Normal Pulled pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h Add eth0 [10.132.0.113/23] from ovn-kubernetes 47m Normal Created pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h Container created 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-1e708b2f-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Started pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h Container started 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-1e708b2f-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Killing pod/payload-processing-e2e-ait-c4e6c558-5f756664fb-gsb6t Stopping container payload-processing 47m Normal Killing pod/payload-pre-processing-e2e-ait-c4e6c558-5f57cf845f-82kds Stopping container payload-pre-processing 46m Normal AddedLabel gateway/e2e-ait-573837e3 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-573837e3 46m Normal ScalingReplicaSet deployment/e2e-ait-573837e3-openshift-default Scaled up replica set e2e-ait-573837e3-openshift-default-7779f764c from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-ait-573837e3-openshift-default-7779f764c Created pod: e2e-ait-573837e3-openshift-default-7779f764c-crrqn 46m Normal Killing pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Stopping container istio-proxy 46m Normal AddedInterface pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Add eth0 [10.132.0.115/23] from ovn-kubernetes 46m Warning Unhealthy pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-migrate-1e708b2f-gw-openshift-default-545cdc848f-lcmj2 Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal Killing pod/e2e-derive-74b07396-openshift-default-58f4bbdb7c-jswjm Stopping container istio-proxy 46m Normal Killing pod/e2e-ait-c4e6c558-openshift-default-7977ff48f9-kxrv4 Stopping container istio-proxy 46m Warning Unhealthy pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused... 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-573837e3-6d645bbdff Error creating: pods "payload-pre-processing-e2e-ait-573837e3-6d645bbdff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-573837e3: serviceaccount "payload-processing-e2e-ait-573837e3" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-573837e3 Scaled up replica set payload-pre-processing-e2e-ait-573837e3-6d645bbdff from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-573837e3 Scaled up replica set payload-processing-e2e-ait-573837e3-65cfb8bddd from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-573837e3-6d645bbdff Created pod: payload-pre-processing-e2e-ait-573837e3-6d645bbdff-dpfv2 46m Warning FailedCreate replicaset/payload-processing-e2e-ait-573837e3-65cfb8bddd Error creating: pods "payload-processing-e2e-ait-573837e3-65cfb8bddd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-573837e3: serviceaccount "payload-processing-e2e-ait-573837e3" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-573837e3-65cfb8bddd Created pod: payload-processing-e2e-ait-573837e3-65cfb8bddd-mqv4g 46m Normal AddedInterface pod/payload-processing-e2e-ait-573837e3-65cfb8bddd-mqv4g Add eth0 [10.134.0.135/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-ait-573837e3-6d645bbdff-dpfv2 Container created 46m Normal Started pod/payload-pre-processing-e2e-ait-573837e3-6d645bbdff-dpfv2 Container started 46m Normal Pulled pod/payload-processing-e2e-ait-573837e3-65cfb8bddd-mqv4g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Pulled pod/payload-pre-processing-e2e-ait-573837e3-6d645bbdff-dpfv2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-ait-573837e3-6d645bbdff-dpfv2 Add eth0 [10.132.0.116/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-ait-573837e3-65cfb8bddd-mqv4g Container started 46m Normal Created pod/payload-processing-e2e-ait-573837e3-65cfb8bddd-mqv4g Container created 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c Created pod: e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v 46m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-60e8f4-openshift-default Scaled up replica set e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c from 0 to 1 46m Normal AddedLabel gateway/e2e-shared-b-w6-60e8f4 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-60e8f4 46m Normal AddedInterface pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Add eth0 [10.133.0.85/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx Container started 46m Normal Created pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx Container created 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-60e8f4 Scaled up replica set payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-60e8f4 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c Created pod: payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c Error creating: pods "payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-60e8f4: serviceaccount "payload-processing-e2e-shared-b-w6-60e8f4" not found 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-573837e3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-573837e3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb Error creating: pods "payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-60e8f4: serviceaccount "payload-processing-e2e-shared-b-w6-60e8f4" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb Created pod: payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 46m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 Add eth0 [10.132.0.118/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 Container started 46m Normal Pulled pod/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 Container started 46m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 Container created 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 Add eth0 [10.134.0.136/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 Container created 46m Warning Unhealthy pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx Startup probe failed: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx Startup probe error: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused... 46m Normal AddedLabel gateway/e2e-mt-d8090513 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d8090513 46m Normal AddedInterface pod/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 Add eth0 [10.132.0.119/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-mt-d8090513-openshift-default-6f56d54b45 Created pod: e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 46m Normal ScalingReplicaSet deployment/e2e-mt-d8090513-openshift-default Scaled up replica set e2e-mt-d8090513-openshift-default-6f56d54b45 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-d8090513 Scaled up replica set payload-processing-e2e-mt-d8090513-69dcffc55b from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-d8090513 Scaled up replica set payload-pre-processing-e2e-mt-d8090513-66c4b9c87 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-mt-d8090513-69dcffc55b Error creating: pods "payload-processing-e2e-mt-d8090513-69dcffc55b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d8090513: serviceaccount "payload-processing-e2e-mt-d8090513" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-d8090513-69dcffc55b Created pod: payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-d8090513-66c4b9c87 Created pod: payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-d8090513-66c4b9c87 Error creating: pods "payload-pre-processing-e2e-mt-d8090513-66c4b9c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d8090513: serviceaccount "payload-processing-e2e-mt-d8090513" not found 46m Normal AddedInterface pod/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 Add eth0 [10.133.0.86/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 Add eth0 [10.134.0.138/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 Container created 46m Normal Pulling pod/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 46m Normal Pulled pod/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.4s (2.4s including waiting). Image size: 125317101 bytes. 46m Normal Started pod/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 Container started 46m Normal Created pod/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 Container created 46m Warning Unhealthy pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Readiness probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Readiness probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal Started pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Container started 46m Normal Created pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Container created 46m Normal Pulled pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Killing pod/payload-pre-processing-e2e-ait-573837e3-6d645bbdff-dpfv2 Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-ait-573837e3-65cfb8bddd-mqv4g Stopping container payload-processing 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-573837e3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-573837e3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning Unhealthy pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 45m Warning ProbeError pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 45m Normal AddedLabel gateway/e2e-shared-a-w0-bea3ef Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-bea3ef 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-bea3ef-openshift-default Scaled up replica set e2e-shared-a-w0-bea3ef-openshift-default-784b4db987 from 0 to 1 45m Normal Killing pod/e2e-ait-573837e3-openshift-default-7779f764c-crrqn Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987 Created pod: e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n 45m Normal AddedLabel gateway/e2e-shared-a-w1-95492e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-95492e 45m Normal AddedInterface pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n Add eth0 [10.134.0.140/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9 Created pod: e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-95492e-openshift-default Scaled up replica set e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9 from 0 to 1 45m Normal AddedInterface pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Add eth0 [10.133.0.87/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-bea3ef Scaled up replica set payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4 Error creating: pods "payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-bea3ef: serviceaccount "payload-processing-e2e-shared-a-w0-bea3ef" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-bea3ef: serviceaccount "payload-processing-e2e-shared-a-w0-bea3ef" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79 Created pod: payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-bea3ef Scaled up replica set payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4 Created pod: payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk Add eth0 [10.132.0.122/23] from ovn-kubernetes 45m Warning ProbeError pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Readiness probe error: Get "http://10.133.0.85:15021/healthz/ready": read tcp 10.133.0.2:44116->10.133.0.85:15021: read: connection reset by peer... 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning Unhealthy pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Readiness probe failed: Get "http://10.133.0.85:15021/healthz/ready": read tcp 10.133.0.2:44116->10.133.0.85:15021: read: connection reset by peer 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk Container created 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-95492e Scaled up replica set payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4 from 0 to 1 45m Normal Created pod/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl Container created 45m Normal Pulled pod/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl Add eth0 [10.132.0.123/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl Container started 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk Container started 45m Normal Created pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Container created 45m Normal Started pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Container started 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4 Created pod: payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4 Error creating: pods "payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-95492e: serviceaccount "payload-processing-e2e-shared-a-w1-95492e" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-95492e: serviceaccount "payload-processing-e2e-shared-a-w1-95492e" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87 Created pod: payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-95492e Scaled up replica set payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87 from 0 to 1 45m Normal Started pod/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w Add eth0 [10.132.0.125/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v Add eth0 [10.132.0.124/23] from ovn-kubernetes 45m Normal Created pod/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v Container created 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w Container created 45m Normal Pulled pod/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w Container started 45m Warning Unhealthy pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-60e8f4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal ScalingReplicaSet deployment/e2e-api-a-6867b8-openshift-default Scaled up replica set e2e-api-a-6867b8-openshift-default-54d69b94c9 from 0 to 1 45m Normal AddedLabel gateway/e2e-api-a-6867b8 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-6867b8 45m Normal SuccessfulCreate replicaset/e2e-api-a-6867b8-openshift-default-54d69b94c9 Created pod: e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-60e8f4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal AddedInterface pod/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd Add eth0 [10.133.0.92/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-6867b8 Scaled up replica set payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75 Error creating: pods "payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-6867b8: serviceaccount "payload-processing-e2e-api-a-6867b8" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-6867b8 Scaled up replica set payload-processing-e2e-api-a-6867b8-757c78757b from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-6867b8-757c78757b Created pod: payload-processing-e2e-api-a-6867b8-757c78757b-7krfg 45m Warning FailedCreate replicaset/payload-processing-e2e-api-a-6867b8-757c78757b Error creating: pods "payload-processing-e2e-api-a-6867b8-757c78757b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-6867b8: serviceaccount "payload-processing-e2e-api-a-6867b8" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75 Created pod: payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-j7xm7 45m Normal Pulled pod/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-j7xm7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-j7xm7 Container created 45m Normal Pulled pod/payload-processing-e2e-api-a-6867b8-757c78757b-7krfg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-api-a-6867b8-757c78757b-7krfg Container created 45m Normal Started pod/payload-processing-e2e-api-a-6867b8-757c78757b-7krfg Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-j7xm7 Add eth0 [10.133.0.93/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-j7xm7 Container started 45m Normal AddedInterface pod/payload-processing-e2e-api-a-6867b8-757c78757b-7krfg Add eth0 [10.134.0.141/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-api-a-353403-openshift-default-5f567587b9 Created pod: e2e-api-a-353403-openshift-default-5f567587b9-q56px 45m Normal AddedLabel gateway/e2e-api-a-353403 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-353403 45m Normal ScalingReplicaSet deployment/e2e-api-a-353403-openshift-default Scaled up replica set e2e-api-a-353403-openshift-default-5f567587b9 from 0 to 1 45m Normal AddedInterface pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Add eth0 [10.134.0.142/23] from ovn-kubernetes 45m Normal Started pod/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 Container started 45m Normal Pulled pod/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 Container created 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-353403-75df964455 Error creating: pods "payload-pre-processing-e2e-api-a-353403-75df964455-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-353403: serviceaccount "payload-processing-e2e-api-a-353403" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-353403-75df964455 Created pod: payload-pre-processing-e2e-api-a-353403-75df964455-rttsx 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-353403 Scaled up replica set payload-pre-processing-e2e-api-a-353403-75df964455 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-353403 Scaled up replica set payload-processing-e2e-api-a-353403-55845f89db from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-353403-55845f89db Created pod: payload-processing-e2e-api-a-353403-55845f89db-zx8nt 45m Warning FailedCreate replicaset/payload-processing-e2e-api-a-353403-55845f89db Error creating: pods "payload-processing-e2e-api-a-353403-55845f89db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-353403: serviceaccount "payload-processing-e2e-api-a-353403" not found 45m Normal Pulled pod/payload-pre-processing-e2e-api-a-353403-75df964455-rttsx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-353403-75df964455-rttsx Add eth0 [10.134.0.143/23] from ovn-kubernetes 45m Normal Created pod/payload-processing-e2e-api-a-353403-55845f89db-zx8nt Container created 45m Normal Pulled pod/payload-processing-e2e-api-a-353403-55845f89db-zx8nt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-api-a-353403-55845f89db-zx8nt Add eth0 [10.134.0.144/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-api-a-353403-75df964455-rttsx Container started 45m Normal Created pod/payload-pre-processing-e2e-api-a-353403-75df964455-rttsx Container created 45m Normal Started pod/payload-processing-e2e-api-a-353403-55845f89db-zx8nt Container started 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-60e8f4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-60e8f4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d8090513-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d8090513-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning Unhealthy pod/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 45m Warning ProbeError pod/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 44m Normal AddedLabel gateway/e2e-shared-b-w0-62f5d1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-62f5d1 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585 Created pod: e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-62f5d1-openshift-default Scaled up replica set e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585 from 0 to 1 44m Normal AddedInterface pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Add eth0 [10.134.0.145/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-api-a-610065-openshift-default-6cbfd66d77 Created pod: e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr 44m Normal ScalingReplicaSet deployment/e2e-api-a-610065-openshift-default Scaled up replica set e2e-api-a-610065-openshift-default-6cbfd66d77 from 0 to 1 44m Normal AddedLabel gateway/e2e-api-a-610065 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-610065 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f Error creating: pods "payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-62f5d1: serviceaccount "payload-processing-e2e-shared-b-w0-62f5d1" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-62f5d1 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f Created pod: payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-62f5d1-6649996454 Error creating: pods "payload-processing-e2e-shared-b-w0-62f5d1-6649996454-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-62f5d1: serviceaccount "payload-processing-e2e-shared-b-w0-62f5d1" not found 44m Normal AddedInterface pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Add eth0 [10.134.0.146/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-62f5d1-6649996454 Created pod: payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-62f5d1 Scaled up replica set payload-processing-e2e-shared-b-w0-62f5d1-6649996454 from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-b-w1-16c1bb Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-16c1bb 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx Add eth0 [10.133.0.97/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj Add eth0 [10.133.0.96/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-16c1bb-openshift-default Scaled up replica set e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 from 0 to 1 44m Normal Started pod/payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx Container started 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q 44m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Add eth0 [10.132.0.128/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-610065 Scaled up replica set payload-processing-e2e-api-a-610065-997f67946 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-610065 Scaled up replica set payload-pre-processing-e2e-api-a-610065-66c456c5f5 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-610065-66c456c5f5 Created pod: payload-pre-processing-e2e-api-a-610065-66c456c5f5-rsst7 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-610065-66c456c5f5 Error creating: pods "payload-pre-processing-e2e-api-a-610065-66c456c5f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-610065: serviceaccount "payload-processing-e2e-api-a-610065" not found 44m Warning FailedCreate replicaset/payload-processing-e2e-api-a-610065-997f67946 Error creating: pods "payload-processing-e2e-api-a-610065-997f67946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-610065: serviceaccount "payload-processing-e2e-api-a-610065" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-610065-997f67946 Created pod: payload-processing-e2e-api-a-610065-997f67946-54k2v 44m Normal Pulled pod/payload-processing-e2e-api-a-610065-997f67946-54k2v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-api-a-610065-66c456c5f5-rsst7 Container created 44m Normal Started pod/payload-pre-processing-e2e-api-a-610065-66c456c5f5-rsst7 Container started 44m Normal Started pod/payload-processing-e2e-api-a-610065-997f67946-54k2v Container started 44m Normal Created pod/payload-processing-e2e-api-a-610065-997f67946-54k2v Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-610065-66c456c5f5-rsst7 Add eth0 [10.133.0.98/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-processing-e2e-api-a-610065-997f67946-54k2v Add eth0 [10.133.0.99/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-api-a-610065-66c456c5f5-rsst7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-16c1bb: serviceaccount "payload-processing-e2e-shared-b-w1-16c1bb" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-16c1bb Scaled up replica set payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74 Error creating: pods "payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-16c1bb: serviceaccount "payload-processing-e2e-shared-b-w1-16c1bb" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-16c1bb Scaled up replica set payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647 Created pod: payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74 Created pod: payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps Container started 44m Normal Created pod/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth Container created 44m Normal Pulled pod/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth Add eth0 [10.133.0.101/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth Container started 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps Add eth0 [10.133.0.100/23] from ovn-kubernetes 44m Normal Started pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n Container started 44m Normal Created pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n Container created 44m Normal AddedLabel gateway/e2e-mt-a4fe0143 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-a4fe0143 44m Normal Pulled pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Pulled pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal SuccessfulCreate replicaset/e2e-mt-a4fe0143-openshift-default-69d54b6bdf Created pod: e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx 44m Normal ScalingReplicaSet deployment/e2e-mt-a4fe0143-openshift-default Scaled up replica set e2e-mt-a4fe0143-openshift-default-69d54b6bdf from 0 to 1 44m Normal Created pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Container created 44m Normal AddedInterface pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Add eth0 [10.134.0.147/23] from ovn-kubernetes 44m Normal Started pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Container started 44m Normal Pulled pod/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d8090513-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d8090513-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Created pod/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd Container created 44m Normal Started pod/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd Container started 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a4fe0143 Scaled up replica set payload-processing-e2e-mt-a4fe0143-6b46b4f5b4 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4 Error creating: pods "payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a4fe0143: serviceaccount "payload-processing-e2e-mt-a4fe0143" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4 Created pod: payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-b22r6 44m Warning Unhealthy pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 44m Warning ProbeError pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a4fe0143-d94c99447 Error creating: pods "payload-pre-processing-e2e-mt-a4fe0143-d94c99447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a4fe0143: serviceaccount "payload-processing-e2e-mt-a4fe0143" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a4fe0143-d94c99447 Created pod: payload-pre-processing-e2e-mt-a4fe0143-d94c99447-4lmxh 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a4fe0143 Scaled up replica set payload-pre-processing-e2e-mt-a4fe0143-d94c99447 from 0 to 1 44m Warning ProbeError pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n Startup probe error: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n Startup probe failed: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused 44m Normal SuccessfulCreate replicaset/e2e-api-b-73b32e-openshift-default-5d4d7cb979 Created pod: e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw 44m Normal Started pod/payload-pre-processing-e2e-mt-a4fe0143-d94c99447-4lmxh Container started 44m Normal AddedInterface pod/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-b22r6 Add eth0 [10.132.0.130/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-b22r6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a4fe0143-d94c99447-4lmxh Add eth0 [10.132.0.129/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-mt-a4fe0143-d94c99447-4lmxh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-b22r6 Container created 44m Normal ScalingReplicaSet deployment/e2e-api-b-73b32e-openshift-default Scaled up replica set e2e-api-b-73b32e-openshift-default-5d4d7cb979 from 0 to 1 44m Normal Started pod/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-b22r6 Container started 44m Normal AddedLabel gateway/e2e-api-b-73b32e Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-73b32e 44m Normal Created pod/payload-pre-processing-e2e-mt-a4fe0143-d94c99447-4lmxh Container created 44m Warning Unhealthy pod/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd Startup probe failed: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused 44m Warning ProbeError pod/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd Startup probe error: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused... 44m Normal AddedInterface pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Add eth0 [10.134.0.148/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-73b32e Scaled up replica set payload-processing-e2e-api-b-73b32e-6655649548 from 0 to 1 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-610065-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667 Error creating: pods "payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-73b32e: serviceaccount "payload-processing-e2e-api-b-73b32e" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-73b32e-6655649548 Created pod: payload-processing-e2e-api-b-73b32e-6655649548-fn9bv 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-73b32e Scaled up replica set payload-pre-processing-e2e-api-b-73b32e-7dd75d8667 from 0 to 1 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-610065-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-jxhrq Add eth0 [10.134.0.149/23] from ovn-kubernetes 44m Warning FailedCreate replicaset/payload-processing-e2e-api-b-73b32e-6655649548 Error creating: pods "payload-processing-e2e-api-b-73b32e-6655649548-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-73b32e: serviceaccount "payload-processing-e2e-api-b-73b32e" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667 Created pod: payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-jxhrq 44m Normal Created pod/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-jxhrq Container created 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-95492e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Started pod/payload-processing-e2e-api-b-73b32e-6655649548-fn9bv Container started 44m Normal Pulled pod/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-jxhrq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-jxhrq Container started 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-95492e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Created pod/payload-processing-e2e-api-b-73b32e-6655649548-fn9bv Container created 44m Normal AddedInterface pod/payload-processing-e2e-api-b-73b32e-6655649548-fn9bv Add eth0 [10.134.0.150/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-api-b-73b32e-6655649548-fn9bv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-6867b8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-6867b8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-bea3ef-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-bea3ef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Deleted pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Deleted pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 7; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-9hgfv Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled down replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 10 to 7 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Deleted pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 44m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-4zpb7 Stopping container istio-proxy 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-95492e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-m7jml Stopping container istio-proxy 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-95492e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-6867b8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-6867b8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulCreate replicaset/e2e-api-a-353403-openshift-default-5f567587b9 Created pod: e2e-api-a-353403-openshift-default-5f567587b9-sfvmj 43m Normal ScalingReplicaSet deployment/e2e-api-a-353403-openshift-default Scaled up replica set e2e-api-a-353403-openshift-default-5f567587b9 from 1 to 2 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-353403-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/e2e-api-b-a32412-openshift-default Scaled up replica set e2e-api-b-a32412-openshift-default-7d4f8c46bf from 0 to 1 43m Normal AddedInterface pod/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj Add eth0 [10.134.0.151/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/e2e-api-b-a32412-openshift-default-7d4f8c46bf Created pod: e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn 43m Normal Pulled pod/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedLabel gateway/e2e-api-b-a32412 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-a32412 43m Normal Started pod/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj Container started 43m Normal Created pod/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj Container created 43m Normal AddedInterface pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Add eth0 [10.134.0.152/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Readiness probe failed: Get "http://10.134.0.145:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Readiness probe error: Get "http://10.134.0.145:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning ProbeError pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Readiness probe error: Get "http://10.134.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Readiness probe failed: Get "http://10.134.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Pulled pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Container started 43m Normal Created pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Container created 43m Normal Pulled pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Container started 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-a32412 Scaled up replica set payload-processing-e2e-api-b-a32412-ccf56585d from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-api-b-a32412-ccf56585d Error creating: pods "payload-processing-e2e-api-b-a32412-ccf56585d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-a32412: serviceaccount "payload-processing-e2e-api-b-a32412" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-a32412 Scaled up replica set payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c Error creating: pods "payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-a32412: serviceaccount "payload-processing-e2e-api-b-a32412" not found 43m Normal Created pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Container created 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c Created pod: payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-xfbc4 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-a32412-ccf56585d Created pod: payload-processing-e2e-api-b-a32412-ccf56585d-ns54m 43m Warning ProbeError pod/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-xfbc4 Add eth0 [10.134.0.154/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-api-b-a32412-ccf56585d-ns54m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-xfbc4 Container created 43m Normal Pulled pod/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-xfbc4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-api-b-a32412-ccf56585d-ns54m Add eth0 [10.134.0.155/23] from ovn-kubernetes 43m Normal Created pod/payload-processing-e2e-api-b-a32412-ccf56585d-ns54m Container created 43m Normal Started pod/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-xfbc4 Container started 43m Normal Started pod/payload-processing-e2e-api-b-a32412-ccf56585d-ns54m Container started 43m Warning Unhealthy pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Startup probe failed: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused 43m Warning ProbeError pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Startup probe error: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused... 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-73b32e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-73b32e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning ProbeError pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused 43m Normal Killing pod/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 Stopping container payload-pre-processing 43m Normal AddedLabel gateway/e2e-api-b-601aea Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-601aea 43m Normal SuccessfulCreate replicaset/e2e-api-b-601aea-openshift-default-8559b84459 Created pod: e2e-api-b-601aea-openshift-default-8559b84459-xs799 43m Normal ScalingReplicaSet deployment/e2e-api-b-601aea-openshift-default Scaled up replica set e2e-api-b-601aea-openshift-default-8559b84459 from 0 to 1 43m Normal AddedInterface pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Add eth0 [10.134.0.158/23] from ovn-kubernetes 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-353403-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-353403-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-601aea Scaled up replica set payload-processing-e2e-api-b-601aea-68cd56fdcb from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-601aea-6587946c4c Created pod: payload-pre-processing-e2e-api-b-601aea-6587946c4c-gzxrq 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-601aea Scaled up replica set payload-pre-processing-e2e-api-b-601aea-6587946c4c from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-api-b-601aea-68cd56fdcb Error creating: pods "payload-processing-e2e-api-b-601aea-68cd56fdcb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-601aea: serviceaccount "payload-processing-e2e-api-b-601aea" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-601aea-68cd56fdcb Created pod: payload-processing-e2e-api-b-601aea-68cd56fdcb-99xxg 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-601aea-6587946c4c Error creating: pods "payload-pre-processing-e2e-api-b-601aea-6587946c4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-601aea: serviceaccount "payload-processing-e2e-api-b-601aea" not found 43m Normal AddedInterface pod/payload-processing-e2e-api-b-601aea-68cd56fdcb-99xxg Add eth0 [10.134.0.160/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-601aea-6587946c4c-gzxrq Add eth0 [10.134.0.159/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-api-b-601aea-6587946c4c-gzxrq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-api-b-601aea-6587946c4c-gzxrq Container created 43m Normal Started pod/payload-pre-processing-e2e-api-b-601aea-6587946c4c-gzxrq Container started 43m Warning Unhealthy pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-sdvw6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 43m Normal Pulled pod/payload-processing-e2e-api-b-601aea-68cd56fdcb-99xxg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-api-b-601aea-68cd56fdcb-99xxg Container started 43m Normal Created pod/payload-processing-e2e-api-b-601aea-68cd56fdcb-99xxg Container created 43m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Readiness probe error: Get "http://10.134.0.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Readiness probe failed: Get "http://10.134.0.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning Unhealthy pod/payload-processing-e2e-mt-d8090513-69dcffc55b-6mxk5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-73b32e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/e2e-api-b-73b32e-openshift-default-5d4d7cb979 Created pod: e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 43m Normal ScalingReplicaSet deployment/e2e-api-b-73b32e-openshift-default Scaled up replica set e2e-api-b-73b32e-openshift-default-5d4d7cb979 from 1 to 4 43m Normal SuccessfulCreate replicaset/e2e-api-b-73b32e-openshift-default-5d4d7cb979 Created pod: e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj 43m Normal SuccessfulCreate replicaset/e2e-api-b-73b32e-openshift-default-5d4d7cb979 Created pod: e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-62f5d1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-62f5d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning ProbeError pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Readiness probe error: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Readiness probe failed: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal SuccessfulCreate replicaset/e2e-mt-a4fe0143-openshift-default-69d54b6bdf Created pod: e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm 43m Normal SuccessfulCreate replicaset/e2e-mt-a4fe0143-openshift-default-69d54b6bdf Created pod: e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a4fe0143-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/e2e-mt-a4fe0143-openshift-default Scaled up replica set e2e-mt-a4fe0143-openshift-default-69d54b6bdf from 1 to 4 43m Normal SuccessfulCreate replicaset/e2e-mt-a4fe0143-openshift-default-69d54b6bdf Created pod: e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd 43m Warning Unhealthy pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Readiness probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 43m Warning ProbeError pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Readiness probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 43m Normal Pulled pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Container created 43m Normal Started pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Container started 43m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Readiness probe error: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Readiness probe failed: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused 43m Normal Pulled pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Container created 43m Normal Started pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Container started 43m Normal Created pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Container created 43m Normal Pulled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Container started 42m Warning ProbeError pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 42m Warning ProbeError pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Startup probe error: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Startup probe failed: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 7; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled down replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 10 to 7 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Deleted pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Deleted pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb 42m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Deleted pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-mj6fk 42m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-s85xb Stopping container istio-proxy 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a4fe0143-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Stopping container istio-proxy 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a4fe0143-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Killing pod/payload-processing-e2e-api-a-6867b8-757c78757b-7krfg Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-api-a-6867b8-d5f6c9b75-j7xm7 Stopping container payload-pre-processing 42m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-vhm2z Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal Created pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 Container created 42m Normal Started pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 Container started 42m Normal Pulled pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 Add eth0 [10.132.0.134/23] from ovn-kubernetes 42m Normal Started pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 Container started 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-73b32e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-73b32e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Created pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 Container created 42m Normal AddedInterface pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 Add eth0 [10.132.0.133/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj Container created 42m Normal Started pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj Container started 42m Normal Pulled pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj Add eth0 [10.134.0.163/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled down replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 10 to 7 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 7; reason: All metrics below target 42m Warning ProbeError pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj Startup probe error: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Readiness probe error: Get "http://10.134.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning ProbeError pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Readiness probe failed: Get "http://10.134.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Deleted pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-6wbpd 42m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Deleted pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs 42m Warning Unhealthy pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj Startup probe failed: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused 42m Warning ProbeError pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Deleted pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-5pjlv 42m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-wgqzs Stopping container istio-proxy 42m Normal Started pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Container started 42m Normal Pulled pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Container created 42m Normal AddedInterface pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm Add eth0 [10.134.0.166/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv Add eth0 [10.134.0.165/23] from ovn-kubernetes 42m Normal Started pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv Container started 42m Normal Pulled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv Container created 42m Normal Pulled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm Container started 42m Normal AddedInterface pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd Add eth0 [10.134.0.164/23] from ovn-kubernetes 42m Normal Started pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd Container started 42m Normal Created pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd Container created 42m Normal Created pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm Container created 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-d8090513-69dcffc55b Created pod: payload-processing-e2e-mt-d8090513-69dcffc55b-xgmzw 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-d8090513 Scaled up replica set payload-processing-e2e-mt-d8090513-69dcffc55b from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-mt-d8090513-69dcffc55b Error creating: pods "payload-processing-e2e-mt-d8090513-69dcffc55b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d8090513: serviceaccount "payload-processing-e2e-mt-d8090513" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-d8090513-66c4b9c87 Created pod: payload-pre-processing-e2e-mt-d8090513-66c4b9c87-5xchw 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-d8090513 Scaled up replica set payload-pre-processing-e2e-mt-d8090513-66c4b9c87 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-d8090513-66c4b9c87 Error creating: pods "payload-pre-processing-e2e-mt-d8090513-66c4b9c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d8090513: serviceaccount "payload-processing-e2e-mt-d8090513" not found 42m Normal Created pod/payload-processing-e2e-mt-d8090513-69dcffc55b-xgmzw Container created 42m Normal Pulled pod/payload-processing-e2e-mt-d8090513-69dcffc55b-xgmzw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning ProbeError pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused 42m Normal AddedInterface pod/payload-processing-e2e-mt-d8090513-69dcffc55b-xgmzw Add eth0 [10.133.0.108/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-5xchw Container started 42m Normal Created pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-5xchw Container created 42m Normal Started pod/payload-processing-e2e-mt-d8090513-69dcffc55b-xgmzw Container started 42m Normal Pulled pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-5xchw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-5xchw Add eth0 [10.133.0.107/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv Startup probe error: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd Startup probe error: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm Startup probe error: Get "http://10.134.0.166:15021/healthz/ready": dial tcp 10.134.0.166:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv Startup probe failed: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-610065-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-610065-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm Startup probe failed: Get "http://10.134.0.166:15021/healthz/ready": dial tcp 10.134.0.166:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd Startup probe failed: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a4fe0143-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a4fe0143-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Readiness probe failed: Get "http://10.134.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Readiness probe error: Get "http://10.134.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Started pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Container started 42m Normal Created pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Container created 42m Normal Pulled pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulDelete replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Deleted pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f 42m Normal SuccessfulDelete replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Deleted pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled down replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 10 to 8 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 8; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-cc8pt Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled down replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 10 to 7 42m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xkg8f Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Deleted pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Deleted pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Deleted pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 7; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kn9f8 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tmb6b Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Deleted pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw 42m Normal ScalingReplicaSet deployment/e2e-mt-a4fe0143-openshift-default Scaled up replica set e2e-mt-a4fe0143-openshift-default-69d54b6bdf from 4 to 7 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a4fe0143-openshift-default New size: 7; reason: 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Deleted pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 42m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-kmrg5 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-gqfmw Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 7; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled down replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 10 to 7 42m Normal SuccessfulCreate replicaset/e2e-mt-a4fe0143-openshift-default-69d54b6bdf Created pod: e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 42m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-kpx7h Stopping container istio-proxy 42m Normal SuccessfulCreate replicaset/e2e-mt-a4fe0143-openshift-default-69d54b6bdf Created pod: e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Deleted pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-cgps2 42m Normal SuccessfulCreate replicaset/e2e-mt-a4fe0143-openshift-default-69d54b6bdf Created pod: e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj 42m Warning ProbeError pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Startup probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Startup probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 41m Normal Created pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 Container created 41m Normal Started pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 Container started 41m Normal Started pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp Container started 41m Normal Created pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp Container created 41m Normal AddedInterface pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 Add eth0 [10.134.0.168/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj Add eth0 [10.134.0.170/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj Container created 41m Normal Started pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj Container started 41m Normal Pulled pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp Add eth0 [10.134.0.169/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 Startup probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp Startup probe failed: Get "http://10.134.0.169:15021/healthz/ready": dial tcp 10.134.0.169:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj Startup probe error: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp Startup probe error: Get "http://10.134.0.169:15021/healthz/ready": dial tcp 10.134.0.169:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj Startup probe failed: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 Startup probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused... 41m Normal Killing pod/payload-pre-processing-e2e-api-a-610065-66c456c5f5-rsst7 Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-api-a-610065-997f67946-54k2v Stopping container payload-processing 41m Normal Killing pod/e2e-api-a-6867b8-openshift-default-54d69b94c9-9v5zd Stopping container istio-proxy 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-601aea-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-601aea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Killing pod/payload-pre-processing-e2e-api-b-73b32e-7dd75d8667-jxhrq Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-api-b-73b32e-6655649548-fn9bv Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-api-a-353403-75df964455-rttsx Stopping container payload-pre-processing 41m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 7; reason: All metrics below target 41m Normal Killing pod/payload-processing-e2e-api-a-353403-55845f89db-zx8nt Stopping container payload-processing 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default Scaled down replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 8 to 7 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Deleted pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-82f6l 41m Normal Killing pod/payload-pre-processing-e2e-mt-a4fe0143-d94c99447-4lmxh Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-mt-a4fe0143-6b46b4f5b4-b22r6 Stopping container payload-processing 41m Normal SuccessfulCreate replicaset/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f Created pod: e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv 41m Normal ScalingReplicaSet deployment/e2e-inf-a-5f4f9b-openshift-default Scaled up replica set e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f from 0 to 1 41m Normal AddedLabel gateway/e2e-inf-a-5f4f9b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-5f4f9b 41m Normal AddedInterface pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Add eth0 [10.132.0.137/23] from ovn-kubernetes 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Deleted pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c 41m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Deleted pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled down replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 10 to 7 41m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-wlp9c Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-z5kr9 Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Deleted pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-lcrlt 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 7; reason: All metrics below target 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544 Error creating: pods "payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-5f4f9b: serviceaccount "payload-processing-e2e-inf-a-5f4f9b" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c Error creating: pods "payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-5f4f9b: serviceaccount "payload-processing-e2e-inf-a-5f4f9b" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c Created pod: payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-5f4f9b Scaled up replica set payload-processing-e2e-inf-a-5f4f9b-57b8fc4544 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-5f4f9b Scaled up replica set payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544 Created pod: payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 41m Normal Started pod/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 Container started 41m Normal Started pod/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx Add eth0 [10.132.0.139/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 Add eth0 [10.132.0.138/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 Container created 41m Normal Created pod/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx Container created 41m Normal Killing pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4gkhp Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-hn5lv Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-qqsbd Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-2cwp5 Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-95dtm Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-49jlj Stopping container istio-proxy 41m Normal AddedLabel gateway/e2e-inf-a-43c02d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-43c02d 41m Normal ScalingReplicaSet deployment/e2e-inf-a-43c02d-openshift-default Scaled up replica set e2e-inf-a-43c02d-openshift-default-57c4b85d79 from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-inf-a-43c02d-openshift-default-57c4b85d79 Created pod: e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv 41m Normal AddedInterface pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Add eth0 [10.132.0.141/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-mt-a4fe0143-openshift-default-69d54b6bdf-4hkcx Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-43c02d-f765c975 Created pod: payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-43c02d-7c8c8448c6 Error creating: pods "payload-processing-e2e-inf-a-43c02d-7c8c8448c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-43c02d: serviceaccount "payload-processing-e2e-inf-a-43c02d" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-43c02d-f765c975 Error creating: pods "payload-pre-processing-e2e-inf-a-43c02d-f765c975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-43c02d: serviceaccount "payload-processing-e2e-inf-a-43c02d" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-43c02d Scaled up replica set payload-processing-e2e-inf-a-43c02d-7c8c8448c6 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-43c02d Scaled up replica set payload-pre-processing-e2e-inf-a-43c02d-f765c975 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-43c02d-7c8c8448c6 Created pod: payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h Add eth0 [10.134.0.176/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h Container started 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn Container started 41m Normal Created pod/payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn Container created 41m Normal Created pod/payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h Container created 41m Normal Pulled pod/payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn Add eth0 [10.134.0.177/23] from ovn-kubernetes 41m Normal Killing pod/payload-processing-e2e-mt-d8090513-69dcffc55b-xgmzw Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-mt-d8090513-66c4b9c87-5xchw Stopping container payload-pre-processing 40m Normal Killing pod/e2e-api-a-610065-openshift-default-6cbfd66d77-bs5jr Stopping container istio-proxy 40m Normal Killing pod/e2e-api-a-353403-openshift-default-5f567587b9-q56px Stopping container istio-proxy 40m Normal Killing pod/e2e-api-a-353403-openshift-default-5f567587b9-sfvmj Stopping container istio-proxy 40m Normal Killing pod/payload-processing-e2e-api-b-601aea-68cd56fdcb-99xxg Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-api-b-601aea-6587946c4c-gzxrq Stopping container payload-pre-processing 40m Normal Killing pod/payload-pre-processing-e2e-api-b-a32412-6b6cc7db6c-xfbc4 Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-api-b-a32412-ccf56585d-ns54m Stopping container payload-processing 40m Normal Killing pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-8rhg8 Stopping container istio-proxy 40m Normal Killing pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-dj2mj Stopping container istio-proxy 40m Normal Killing pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xfr55 Stopping container istio-proxy 40m Normal Killing pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Stopping container istio-proxy 40m Normal AddedLabel gateway/e2e-inf-a-780077 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-780077 40m Normal ScalingReplicaSet deployment/e2e-inf-a-780077-openshift-default Scaled up replica set e2e-inf-a-780077-openshift-default-7896f6cfc from 0 to 1 40m Normal SuccessfulCreate replicaset/e2e-inf-a-780077-openshift-default-7896f6cfc Created pod: e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm 40m Normal AddedInterface pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Add eth0 [10.134.0.191/23] from ovn-kubernetes 40m Warning Unhealthy pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-api-b-73b32e-openshift-default-5d4d7cb979-xz9qw Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-780077-667bdfc946 Created pod: payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-780077-667bdfc946 Error creating: pods "payload-processing-e2e-inf-a-780077-667bdfc946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-780077: serviceaccount "payload-processing-e2e-inf-a-780077" not found 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-780077 Scaled up replica set payload-processing-e2e-inf-a-780077-667bdfc946 from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-780077 Scaled up replica set payload-pre-processing-e2e-inf-a-780077-8fb65bb75 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-780077-8fb65bb75 Created pod: payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-780077-8fb65bb75 Error creating: pods "payload-pre-processing-e2e-inf-a-780077-8fb65bb75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-780077: serviceaccount "payload-processing-e2e-inf-a-780077" not found 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl Add eth0 [10.134.0.192/23] from ovn-kubernetes 40m Normal Started pod/payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl Container started 40m Normal Started pod/payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh Container started 40m Normal Pulled pod/payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh Container created 40m Normal Pulled pod/payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl Container created 40m Normal AddedInterface pod/payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh Add eth0 [10.134.0.193/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/e2e-inf-a-820c04-openshift-default-bc774f4f Created pod: e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 40m Normal ScalingReplicaSet deployment/e2e-inf-a-820c04-openshift-default Scaled up replica set e2e-inf-a-820c04-openshift-default-bc774f4f from 0 to 1 40m Normal AddedInterface pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Add eth0 [10.134.0.194/23] from ovn-kubernetes 40m Normal AddedLabel gateway/e2e-inf-a-820c04 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-820c04 40m Normal AddedInterface pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 Add eth0 [10.134.0.195/23] from ovn-kubernetes 40m Normal ScalingReplicaSet deployment/e2e-inf-b-a07920-openshift-default Scaled up replica set e2e-inf-b-a07920-openshift-default-5f4bd748c from 0 to 1 40m Normal AddedLabel gateway/e2e-inf-b-a07920 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-a07920 40m Normal SuccessfulCreate replicaset/e2e-inf-b-a07920-openshift-default-5f4bd748c Created pod: e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc Error creating: pods "payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-820c04: serviceaccount "payload-processing-e2e-inf-a-820c04" not found 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-820c04 Scaled up replica set payload-processing-e2e-inf-a-820c04-7b94544c4b from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-820c04-7b94544c4b Created pod: payload-processing-e2e-inf-a-820c04-7b94544c4b-5sgbd 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-820c04-7b94544c4b Error creating: pods "payload-processing-e2e-inf-a-820c04-7b94544c4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-820c04: serviceaccount "payload-processing-e2e-inf-a-820c04" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-820c04 Scaled up replica set payload-pre-processing-e2e-inf-a-820c04-5cd68568bc from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc Created pod: payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd 40m Normal Pulled pod/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd Container created 40m Normal Created pod/payload-processing-e2e-inf-a-820c04-7b94544c4b-5sgbd Container created 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd Add eth0 [10.134.0.197/23] from ovn-kubernetes 40m Normal Started pod/payload-processing-e2e-inf-a-820c04-7b94544c4b-5sgbd Container started 40m Normal AddedInterface pod/payload-processing-e2e-inf-a-820c04-7b94544c4b-5sgbd Add eth0 [10.134.0.196/23] from ovn-kubernetes 40m Normal Started pod/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd Container started 40m Normal Pulled pod/payload-processing-e2e-inf-a-820c04-7b94544c4b-5sgbd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-a07920 Scaled up replica set payload-processing-e2e-inf-b-a07920-5657fc66dd from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-a07920-5657fc66dd Created pod: payload-processing-e2e-inf-b-a07920-5657fc66dd-vk2lw 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-a07920-6f78447f99 Error creating: pods "payload-pre-processing-e2e-inf-b-a07920-6f78447f99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-a07920: serviceaccount "payload-processing-e2e-inf-b-a07920" not found 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-ea2547-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-ea2547-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-a07920-6f78447f99 Created pod: payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-a07920 Scaled up replica set payload-pre-processing-e2e-inf-b-a07920-6f78447f99 from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-a07920-5657fc66dd Error creating: pods "payload-processing-e2e-inf-b-a07920-5657fc66dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-a07920: serviceaccount "payload-processing-e2e-inf-b-a07920" not found 39m Normal Created pod/payload-processing-e2e-inf-b-a07920-5657fc66dd-vk2lw Container created 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc Add eth0 [10.134.0.198/23] from ovn-kubernetes 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-601aea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-601aea-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Started pod/payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc Container started 39m Normal Pulled pod/payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc Container created 39m Normal AddedInterface pod/payload-processing-e2e-inf-b-a07920-5657fc66dd-vk2lw Add eth0 [10.134.0.199/23] from ovn-kubernetes 39m Normal Started pod/payload-processing-e2e-inf-b-a07920-5657fc66dd-vk2lw Container started 39m Normal Pulled pod/payload-processing-e2e-inf-b-a07920-5657fc66dd-vk2lw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Pulled pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Container created 39m Normal Started pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Container started 39m Normal Pulled pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Container started 39m Normal Created pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Container created 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-62f5d1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-62f5d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal AddedInterface pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Add eth0 [10.134.0.200/23] from ovn-kubernetes 39m Normal ScalingReplicaSet deployment/e2e-inf-b-2a8b26-openshift-default Scaled up replica set e2e-inf-b-2a8b26-openshift-default-84b87df95f from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-inf-b-2a8b26-openshift-default-84b87df95f Created pod: e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf 39m Normal AddedLabel gateway/e2e-inf-b-2a8b26 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-2a8b26 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-a32412-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/e2e-mt-d8090513-openshift-default-6f56d54b45-xstc5 Stopping container istio-proxy 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-a32412-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-5f4f9b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-5f4f9b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning Unhealthy pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Startup probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Startup probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-bea3ef-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-bea3ef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning ProbeError pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Startup probe error: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Startup probe failed: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d Error creating: pods "payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2a8b26: serviceaccount "payload-processing-e2e-inf-b-2a8b26" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-2a8b26 Scaled up replica set payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d Created pod: payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-25x6n 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff Error creating: pods "payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2a8b26: serviceaccount "payload-processing-e2e-inf-b-2a8b26" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff Created pod: payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-2a8b26 Scaled up replica set payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff from 0 to 1 39m Normal Created pod/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-25x6n Container created 39m Normal Created pod/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz Container created 39m Normal AddedInterface pod/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-25x6n Add eth0 [10.134.0.203/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-25x6n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-25x6n Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz Add eth0 [10.134.0.202/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz Container started 39m Normal Pulled pod/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-43c02d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f Created pod: e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt 39m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-969b92-openshift-default Scaled up replica set e2e-shared-a-w2-969b92-openshift-default-b496c6b6f from 0 to 1 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-43c02d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal AddedLabel gateway/e2e-shared-a-w2-969b92 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-969b92 39m Normal AddedInterface pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Add eth0 [10.132.0.143/23] from ovn-kubernetes 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-969b92 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-969b92 Scaled up replica set payload-processing-e2e-shared-a-w2-969b92-75f748cc6b from 0 to 1 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 Add eth0 [10.134.0.204/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47 Created pod: payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-969b92: serviceaccount "payload-processing-e2e-shared-a-w2-969b92" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b Created pod: payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b Error creating: pods "payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-969b92: serviceaccount "payload-processing-e2e-shared-a-w2-969b92" not found 39m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 Container started 39m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 Container created 39m Normal Started pod/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq Container started 39m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq Add eth0 [10.134.0.205/23] from ovn-kubernetes 39m Normal Created pod/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq Container created 39m Normal Pulled pod/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-43c02d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-43c02d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2a8b26-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2a8b26-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-5f4f9b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/e2e-api-b-601aea-openshift-default-8559b84459-xs799 Stopping container istio-proxy 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-5f4f9b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal AddedLabel gateway/e2e-inf-a-49dfaf Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-49dfaf 39m Normal ScalingReplicaSet deployment/e2e-inf-a-49dfaf-openshift-default Scaled up replica set e2e-inf-a-49dfaf-openshift-default-7967d4cccf from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-inf-a-49dfaf-openshift-default-7967d4cccf Created pod: e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 39m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Deleted pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b 39m Normal SuccessfulDelete replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Deleted pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 6; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled down replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 7 to 6 39m Normal SuccessfulDelete replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Deleted pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn 39m Normal SuccessfulDelete replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Deleted pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jtdpt 39m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-lk4pn Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled down replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 7 to 6 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 6; reason: All metrics below target 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 5; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled down replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 7 to 5 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Deleted pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m 39m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-dfz2m Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled down replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 7 to 6 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default New size: 6; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-xh96b Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 6; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-7lxvp Stopping container istio-proxy 39m Normal AddedInterface pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Add eth0 [10.132.0.145/23] from ovn-kubernetes 39m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Deleted pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-hph22 39m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Deleted pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-lxbkd 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Deleted pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 6; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled down replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 7 to 6 39m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kd9xl Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled down replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 7 to 6 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 6; reason: All metrics below target 39m Normal Created pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Container created 39m Normal Pulled pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Container started 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-49dfaf-777485656c Error creating: pods "payload-pre-processing-e2e-inf-a-49dfaf-777485656c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-49dfaf: serviceaccount "payload-processing-e2e-inf-a-49dfaf" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-49dfaf Scaled up replica set payload-pre-processing-e2e-inf-a-49dfaf-777485656c from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-49dfaf Scaled up replica set payload-processing-e2e-inf-a-49dfaf-b8746c684 from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-49dfaf-b8746c684 Error creating: pods "payload-processing-e2e-inf-a-49dfaf-b8746c684-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-49dfaf: serviceaccount "payload-processing-e2e-inf-a-49dfaf" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-49dfaf-b8746c684 Created pod: payload-processing-e2e-inf-a-49dfaf-b8746c684-mpl56 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-49dfaf-777485656c Created pod: payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn Add eth0 [10.133.0.118/23] from ovn-kubernetes 39m Normal Killing pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Stopping container istio-proxy 39m Normal AddedInterface pod/payload-processing-e2e-inf-a-49dfaf-b8746c684-mpl56 Add eth0 [10.132.0.146/23] from ovn-kubernetes 39m Normal AddedLabel gateway/e2e-inf-a-cf3466 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-cf3466 39m Normal Created pod/payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn Container created 39m Normal Pulled pod/payload-processing-e2e-inf-a-49dfaf-b8746c684-mpl56 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal ScalingReplicaSet deployment/e2e-inf-a-cf3466-openshift-default Scaled up replica set e2e-inf-a-cf3466-openshift-default-97cd47b87 from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-inf-a-cf3466-openshift-default-97cd47b87 Created pod: e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c 39m Normal Started pod/payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn Container started 39m Normal Created pod/payload-processing-e2e-inf-a-49dfaf-b8746c684-mpl56 Container created 39m Normal Started pod/payload-processing-e2e-inf-a-49dfaf-b8746c684-mpl56 Container started 39m Normal Pulled pod/payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning ProbeError pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Startup probe error: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused... 39m Normal AddedInterface pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Add eth0 [10.134.0.207/23] from ovn-kubernetes 39m Warning Unhealthy pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Startup probe failed: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused 39m Warning ProbeError pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-api-b-a32412-openshift-default-7d4f8c46bf-2z9gn Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-820c04-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-820c04-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867 Created pod: payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867 Error creating: pods "payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-cf3466: serviceaccount "payload-processing-e2e-inf-a-cf3466" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-cf3466-5858df687b Created pod: payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-cf3466-5858df687b Error creating: pods "payload-processing-e2e-inf-a-cf3466-5858df687b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-cf3466: serviceaccount "payload-processing-e2e-inf-a-cf3466" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-cf3466 Scaled up replica set payload-processing-e2e-inf-a-cf3466-5858df687b from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-cf3466 Scaled up replica set payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867 from 0 to 1 39m Normal Pulled pod/payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Deleted pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 5; reason: All metrics below target 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 5; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled down replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 6 to 5 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf Add eth0 [10.134.0.209/23] from ovn-kubernetes 39m Normal Started pod/payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr Container started 39m Normal Pulled pod/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf Container started 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Deleted pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-lbpf7 39m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-r7ggw Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default Scaled down replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 6 to 5 39m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Deleted pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-2zx9p 39m Normal Created pod/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf Container created 39m Normal Created pod/payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr Container created 39m Normal AddedInterface pod/payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr Add eth0 [10.133.0.119/23] from ovn-kubernetes 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Deleted pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq 39m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-hwhbq Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 5; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled down replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 6 to 5 38m Normal ScalingReplicaSet deployment/e2e-inf-b-1da7b2-openshift-default Scaled up replica set e2e-inf-b-1da7b2-openshift-default-6997bcc7b7 from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7 Created pod: e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj 38m Normal AddedLabel gateway/e2e-inf-b-1da7b2 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-1da7b2 38m Normal AddedInterface pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Add eth0 [10.132.0.147/23] from ovn-kubernetes 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-49dfaf-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-49dfaf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-1da7b2 Scaled up replica set payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b Created pod: payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b Error creating: pods "payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1da7b2: serviceaccount "payload-processing-e2e-inf-b-1da7b2" not found 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675 Error creating: pods "payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1da7b2: serviceaccount "payload-processing-e2e-inf-b-1da7b2" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675 Created pod: payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-1da7b2 Scaled up replica set payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675 from 0 to 1 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt Add eth0 [10.133.0.121/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt Container created 38m Normal Started pod/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt Container started 38m Normal AddedInterface pod/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd Add eth0 [10.132.0.148/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd Container started 38m Normal Pulled pod/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd Container created 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-780077-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-780077-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning ProbeError pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Readiness probe error: Get "http://10.134.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Readiness probe failed: Get "http://10.134.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Readiness probe failed: Get "http://10.134.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Readiness probe error: Get "http://10.134.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Pulled pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 Container started 38m Normal Created pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 Container created 38m Normal ScalingReplicaSet deployment/e2e-inf-b-dd257f-openshift-default Scaled up replica set e2e-inf-b-dd257f-openshift-default-748b4d89df from 0 to 1 38m Normal Created pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Container created 38m Normal AddedLabel gateway/e2e-inf-b-dd257f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-dd257f 38m Normal SuccessfulCreate replicaset/e2e-inf-b-dd257f-openshift-default-748b4d89df Created pod: e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr 38m Normal Started pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Container started 38m Normal Created pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Container created 38m Normal Pulled pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Container started 38m Normal Pulled pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr Add eth0 [10.132.0.150/23] from ovn-kubernetes 38m Warning Unhealthy pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 Startup probe failed: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 Startup probe error: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Startup probe error: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused 38m Warning Unhealthy pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Startup probe failed: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-dd257f Scaled up replica set payload-processing-e2e-inf-b-dd257f-86d8696c74 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-dd257f Scaled up replica set payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7 from 0 to 1 38m Normal AddedInterface pod/payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr Add eth0 [10.133.0.124/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7 Created pod: payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7 Error creating: pods "payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-dd257f: serviceaccount "payload-processing-e2e-inf-b-dd257f" not found 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-dd257f-86d8696c74 Created pod: payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-dd257f-86d8696c74 Error creating: pods "payload-processing-e2e-inf-b-dd257f-86d8696c74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-dd257f: serviceaccount "payload-processing-e2e-inf-b-dd257f" not found 38m Normal Pulled pod/payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr Container started 38m Normal Created pod/payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr Container created 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb Add eth0 [10.133.0.123/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb Container created 38m Normal Started pod/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb Container started 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-820c04-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-820c04-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal AddedLabel gateway/e2e-shared-b-w2-ae06fe Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-ae06fe 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae06fe-openshift-default Scaled up replica set e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 from 0 to 1 37m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Add eth0 [10.134.0.211/23] from ovn-kubernetes 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2a8b26-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2a8b26-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning Unhealthy pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Readiness probe failed: Get "http://10.134.0.207:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Readiness probe error: Get "http://10.134.0.207:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Normal Pulled pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-ae06fe Scaled up replica set payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb from 0 to 1 37m Normal Started pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Container started 37m Normal Created pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Container created 37m Warning Unhealthy pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Readiness probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Readiness probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 37m Normal Pulled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Container started 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-ae06fe: serviceaccount "payload-processing-e2e-shared-b-w2-ae06fe" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55 Created pod: payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 37m Normal Created pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Container created 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb Error creating: pods "payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-ae06fe: serviceaccount "payload-processing-e2e-shared-b-w2-ae06fe" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-ae06fe Scaled up replica set payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb Created pod: payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn 37m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 Container created 37m Normal Pulled pod/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn Container created 37m Normal Started pod/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn Container started 37m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn Add eth0 [10.133.0.126/23] from ovn-kubernetes 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 Add eth0 [10.134.0.212/23] from ovn-kubernetes 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-a07920-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 Container started 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-a07920-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning ProbeError pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Startup probe error: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Startup probe failed: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 37m Normal ScalingReplicaSet deployment/e2e-inf-b-97ceef-openshift-default Scaled up replica set e2e-inf-b-97ceef-openshift-default-77d68d9ff9 from 0 to 1 37m Normal AddedLabel gateway/e2e-inf-b-97ceef Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-97ceef 37m Normal SuccessfulCreate replicaset/e2e-inf-b-97ceef-openshift-default-77d68d9ff9 Created pod: e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 37m Normal AddedInterface pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Add eth0 [10.134.0.213/23] from ovn-kubernetes 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-cf3466-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-cf3466-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-97ceef Scaled up replica set payload-processing-e2e-inf-b-97ceef-684cfb864 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f Error creating: pods "payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-97ceef: serviceaccount "payload-processing-e2e-inf-b-97ceef" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-97ceef Scaled up replica set payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-97ceef-684cfb864 Created pod: payload-processing-e2e-inf-b-97ceef-684cfb864-gxprj 37m Normal Pulled pod/payload-processing-e2e-inf-b-97ceef-684cfb864-gxprj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h Add eth0 [10.133.0.127/23] from ovn-kubernetes 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-97ceef-684cfb864 Error creating: pods "payload-processing-e2e-inf-b-97ceef-684cfb864-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-97ceef: serviceaccount "payload-processing-e2e-inf-b-97ceef" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f Created pod: payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s 37m Normal Created pod/payload-processing-e2e-inf-b-97ceef-684cfb864-gxprj Container created 37m Normal SuccessfulCreate replicaset/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f Created pod: e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h 37m Normal ScalingReplicaSet deployment/e2e-inf-a-5f4f9b-openshift-default Scaled up replica set e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f from 1 to 2 37m Normal Started pod/payload-processing-e2e-inf-b-97ceef-684cfb864-gxprj Container started 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-5f4f9b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal AddedInterface pod/payload-processing-e2e-inf-b-97ceef-684cfb864-gxprj Add eth0 [10.134.0.215/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h Container started 37m Normal Started pod/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s Container started 37m Normal Created pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h Container created 37m Normal Pulled pod/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s Container created 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s Add eth0 [10.132.0.154/23] from ovn-kubernetes 37m Normal Created pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Container created 37m Normal Started pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Container started 37m Normal Pulled pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h Startup probe failed: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused 37m Warning ProbeError pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h Startup probe error: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused... 37m Normal SuccessfulCreate replicaset/e2e-inf-b-f96ede-openshift-default-684654576c Created pod: e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 37m Normal AddedLabel gateway/e2e-inf-b-f96ede Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-f96ede 37m Normal ScalingReplicaSet deployment/e2e-inf-b-f96ede-openshift-default Scaled up replica set e2e-inf-b-f96ede-openshift-default-684654576c from 0 to 1 37m Normal AddedInterface pod/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 Add eth0 [10.132.0.155/23] from ovn-kubernetes 37m Warning Unhealthy pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Startup probe failed: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-cf3466-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-cf3466-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning ProbeError pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Startup probe error: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused... 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79 Error creating: pods "payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f96ede: serviceaccount "payload-processing-e2e-inf-b-f96ede" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-f96ede Scaled up replica set payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79 from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-f96ede Scaled up replica set payload-processing-e2e-inf-b-f96ede-55bbbb7db8 from 0 to 1 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-f96ede-55bbbb7db8 Error creating: pods "payload-processing-e2e-inf-b-f96ede-55bbbb7db8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f96ede: serviceaccount "payload-processing-e2e-inf-b-f96ede" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-f96ede-55bbbb7db8 Created pod: payload-processing-e2e-inf-b-f96ede-55bbbb7db8-956nk 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79 Created pod: payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r Add eth0 [10.133.0.128/23] from ovn-kubernetes 37m Normal Started pod/payload-processing-e2e-inf-b-f96ede-55bbbb7db8-956nk Container started 37m Normal Pulled pod/payload-processing-e2e-inf-b-f96ede-55bbbb7db8-956nk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-inf-b-f96ede-55bbbb7db8-956nk Add eth0 [10.133.0.129/23] from ovn-kubernetes 37m Normal Created pod/payload-processing-e2e-inf-b-f96ede-55bbbb7db8-956nk Container created 37m Normal Pulled pod/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r Container started 37m Normal Created pod/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r Container created 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-969b92-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-969b92-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal Pulled pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr Container created 37m Normal Started pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr Container started 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-dd257f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-dd257f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning ProbeError pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr Startup probe error: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr Startup probe failed: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused 36m Normal ScalingReplicaSet deployment/e2e-inf-a-43c02d-openshift-default Scaled up replica set e2e-inf-a-43c02d-openshift-default-57c4b85d79 from 1 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-43c02d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal AddedInterface pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Add eth0 [10.134.0.216/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/e2e-inf-a-43c02d-openshift-default-57c4b85d79 Created pod: e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t 36m Normal Pulled pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Container created 36m Normal Started pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Container started 36m Normal SuccessfulCreate replicaset/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f Created pod: e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-5f4f9b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 36m Normal ScalingReplicaSet deployment/e2e-inf-a-5f4f9b-openshift-default Scaled up replica set e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f from 2 to 3 36m Normal AddedInterface pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 Add eth0 [10.134.0.217/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal SuccessfulCreate replicaset/e2e-inf-b-a07920-openshift-default-5f4bd748c Created pod: e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 36m Normal ScalingReplicaSet deployment/e2e-inf-b-a07920-openshift-default Scaled up replica set e2e-inf-b-a07920-openshift-default-5f4bd748c from 1 to 2 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-49dfaf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-a07920-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-49dfaf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Started pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 Container started 36m Normal Created pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 Container created 36m Normal AddedInterface pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 Add eth0 [10.133.0.131/23] from ovn-kubernetes 36m Normal Created pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 Container created 36m Normal Pulled pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 Container started 36m Warning ProbeError pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Startup probe error: Get "http://10.134.0.216:15021/healthz/ready": dial tcp 10.134.0.216:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Startup probe failed: Get "http://10.134.0.216:15021/healthz/ready": dial tcp 10.134.0.216:15021: connect: connection refused 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-dd257f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-dd257f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning Unhealthy pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 Startup probe failed: Get "http://10.134.0.217:15021/healthz/ready": dial tcp 10.134.0.217:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 Startup probe error: Get "http://10.134.0.217:15021/healthz/ready": dial tcp 10.134.0.217:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 Startup probe error: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 Startup probe failed: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae06fe-openshift-default Scaled up replica set e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 from 1 to 4 36m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 Add eth0 [10.133.0.134/23] from ovn-kubernetes 36m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn Add eth0 [10.133.0.133/23] from ovn-kubernetes 36m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn Container created 36m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 Container started 36m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn Container started 36m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 Container created 36m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp Add eth0 [10.134.0.219/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Readiness probe failed: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp Container created 36m Warning ProbeError pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Readiness probe error: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp Container started 36m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Readiness probe failed: Get "http://10.134.0.211:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Readiness probe error: Get "http://10.134.0.211:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn Startup probe failed: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 Startup probe error: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 Startup probe failed: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn Startup probe error: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp Startup probe error: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp Startup probe failed: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused 36m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Container created 36m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Container started 36m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Pulled pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Container started 36m Normal Created pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Container created 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-97ceef-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-97ceef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Startup probe failed: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Startup probe error: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Startup probe failed: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Startup probe error: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused... 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-97ceef-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae06fe-openshift-default Scaled up replica set e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 from 4 to 7 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-97ceef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default New size: 7; reason: All metrics below target 35m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f Add eth0 [10.134.0.220/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj Add eth0 [10.133.0.136/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj Add eth0 [10.133.0.137/23] from ovn-kubernetes 35m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj Container created 35m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f Container started 35m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj Container created 35m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f Container created 35m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj Container started 35m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj Container started 35m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f Startup probe failed: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f Startup probe error: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj Startup probe error: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj Startup probe failed: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-dd257f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal ScalingReplicaSet deployment/e2e-inf-b-dd257f-openshift-default Scaled up replica set e2e-inf-b-dd257f-openshift-default-748b4d89df from 1 to 2 35m Normal SuccessfulCreate replicaset/e2e-inf-b-dd257f-openshift-default-748b4d89df Created pod: e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k 35m Normal Started pod/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 Container started 35m Normal AddedInterface pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Add eth0 [10.133.0.138/23] from ovn-kubernetes 35m Normal Created pod/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 Container created 35m Normal Created pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Container created 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f96ede-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Started pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Container started 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f96ede-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Pulled pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Container created 35m Normal Started pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Container started 35m Warning ProbeError pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Startup probe error: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused... 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-1da7b2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-1da7b2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning Unhealthy pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Startup probe failed: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 Startup probe failed: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 Startup probe error: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Startup probe failed: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Startup probe error: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused... 35m Normal SuccessfulCreate replicaset/e2e-inf-b-a07920-openshift-default-5f4bd748c Created pod: e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 35m Normal ScalingReplicaSet deployment/e2e-inf-b-a07920-openshift-default Scaled up replica set e2e-inf-b-a07920-openshift-default-5f4bd748c from 2 to 3 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-a07920-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 34m Normal SuccessfulCreate replicaset/e2e-inf-a-780077-openshift-default-7896f6cfc Created pod: e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-780077-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal ScalingReplicaSet deployment/e2e-inf-a-780077-openshift-default Scaled up replica set e2e-inf-a-780077-openshift-default-7896f6cfc from 1 to 2 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-969b92-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-969b92-openshift-default Scaled up replica set e2e-shared-a-w2-969b92-openshift-default-b496c6b6f from 1 to 2 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f Created pod: e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz 34m Normal ScalingReplicaSet deployment/e2e-inf-a-820c04-openshift-default Scaled up replica set e2e-inf-a-820c04-openshift-default-bc774f4f from 1 to 2 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-820c04-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal SuccessfulCreate replicaset/e2e-inf-a-820c04-openshift-default-bc774f4f Created pod: e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq 34m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default Scaled down replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 6 to 4 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 4; reason: All metrics below target 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 5; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Deleted pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Deleted pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq 34m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-wfwdq Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Deleted pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-ptzw2 34m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled down replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 6 to 5 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f96ede-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f96ede-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Created pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx Container created 34m Normal AddedInterface pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 Add eth0 [10.133.0.142/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Readiness probe error: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Normal Pulled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Add eth0 [10.133.0.144/23] from ovn-kubernetes 34m Normal AddedInterface pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx Add eth0 [10.133.0.143/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-5t7st Readiness probe failed: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Normal Started pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx Container started 34m Normal Started pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 Container started 34m Normal Created pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 Container created 34m Normal Pulled pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Container started 34m Normal Created pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Container created 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-1da7b2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-1da7b2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning Unhealthy pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 Startup probe failed: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 Startup probe error: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused... 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 3; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Deleted pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 34m Normal SuccessfulDelete replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Deleted pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z 34m Normal SuccessfulDelete replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Deleted pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb 34m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default Scaled down replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 5 to 3 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Deleted pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm 34m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Deleted pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 4; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Deleted pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-s5bpf 34m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-68l64 Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default Scaled down replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 5 to 4 34m Normal SuccessfulDelete replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Deleted pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-jk4x8 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 4; reason: All metrics below target 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default New size: 4; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-ndw8z Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 4; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-jj5wb Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-ngzcm Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled down replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 5 to 4 34m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Stopping container istio-proxy 34m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Readiness probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-g2c9p Readiness probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 34m Normal AddedInterface pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq Add eth0 [10.134.0.221/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq Container created 34m Normal Started pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq Container started 34m Warning ProbeError pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq Startup probe error: Get "http://10.134.0.221:15021/healthz/ready": dial tcp 10.134.0.221:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq Startup probe failed: Get "http://10.134.0.221:15021/healthz/ready": dial tcp 10.134.0.221:15021: connect: connection refused 34m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 3; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Deleted pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-gml7x 34m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Deleted pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-b8rm4 34m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default Scaled down replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 4 to 3 34m Normal SuccessfulDelete replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Deleted pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-vx4jp 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj 34m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7bdc57c4cf from 10 to 7 34m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm 34m Normal Killing pod/maas-default-gateway-openshift-default-7bdc57c4cf-p4gkj Stopping container istio-proxy 34m Normal Killing pod/maas-default-gateway-openshift-default-7bdc57c4cf-wzsbm Stopping container istio-proxy 34m Normal Killing pod/maas-default-gateway-openshift-default-7bdc57c4cf-cmb2h Stopping container istio-proxy 33m Normal Killing pod/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx Stopping container payload-pre-processing 33m Normal Killing pod/payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn Stopping container payload-processing 33m Normal Killing pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-lp226 Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-fhw7h Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-a-5f4f9b-openshift-default-5fbb6b5f5f-6cggv Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/payload-processing-e2e-inf-a-5f4f9b-57b8fc4544-pbpc9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-5f4f9b-794b475b4c-hkwrx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-inf-a-43c02d-7c8c8448c6-4bbfn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-43c02d-f765c975-f6d2h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f 33m Normal Killing pod/maas-default-gateway-openshift-default-7bdc57c4cf-ldp9f Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7bdc57c4cf from 7 to 6 33m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 33m Normal Started pod/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 Container started 33m Normal AddedInterface pod/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 Add eth0 [10.134.0.222/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 Container created 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-f96ede-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal ScalingReplicaSet deployment/e2e-inf-b-f96ede-openshift-default Scaled up replica set e2e-inf-b-f96ede-openshift-default-684654576c from 1 to 2 33m Normal SuccessfulCreate replicaset/e2e-inf-b-f96ede-openshift-default-684654576c Created pod: e2e-inf-b-f96ede-openshift-default-684654576c-whs76 33m Normal Killing pod/payload-processing-e2e-inf-b-a07920-5657fc66dd-vk2lw Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc Stopping container payload-pre-processing 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh 33m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n Add eth0 [10.132.0.165/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 33m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae06fe-openshift-default Scaled up replica set e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 from 7 to 10 33m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 (combined from similar events): Created pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n 33m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 Add eth0 [10.132.0.164/23] from ovn-kubernetes 33m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh Container started 33m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 Container created 33m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n Container started 33m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n Container created 33m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 Container started 33m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh Add eth0 [10.132.0.166/23] from ovn-kubernetes 33m Normal Created pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh Container created 33m Normal Pulled pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f Created pod: e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f Created pod: e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-969b92-openshift-default Scaled up replica set e2e-shared-a-w2-969b92-openshift-default-b496c6b6f from 2 to 4 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-969b92-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 33m Normal ScalingReplicaSet deployment/e2e-inf-b-1da7b2-openshift-default Scaled up replica set e2e-inf-b-1da7b2-openshift-default-6997bcc7b7 from 1 to 2 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-1da7b2-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal Started pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Container started 33m Normal Created pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Container created 33m Normal SuccessfulCreate replicaset/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7 Created pod: e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 33m Normal Pulled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Add eth0 [10.134.0.224/23] from ovn-kubernetes 33m Normal AddedInterface pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b Add eth0 [10.132.0.167/23] from ovn-kubernetes 33m Normal Killing pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Stopping container istio-proxy 33m Normal AddedInterface pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 Add eth0 [10.134.0.225/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning ProbeError pod/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 Startup probe error: Get "http://10.134.0.222:15021/healthz/ready": dial tcp 10.134.0.222:15021: connect: connection refused... 33m Normal Pulled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 Container started 33m Normal Created pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 Container created 33m Warning Unhealthy pod/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 Startup probe failed: Get "http://10.134.0.222:15021/healthz/ready": dial tcp 10.134.0.222:15021: connect: connection refused 33m Normal Started pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b Container started 33m Normal Created pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b Container created 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-49dfaf-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal ScalingReplicaSet deployment/e2e-inf-a-49dfaf-openshift-default Scaled up replica set e2e-inf-a-49dfaf-openshift-default-7967d4cccf from 1 to 2 33m Normal ScalingReplicaSet deployment/e2e-inf-b-a07920-openshift-default Scaled up replica set e2e-inf-b-a07920-openshift-default-5f4bd748c from 3 to 4 33m Normal SuccessfulCreate replicaset/e2e-inf-b-a07920-openshift-default-5f4bd748c Created pod: e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z 33m Normal SuccessfulCreate replicaset/e2e-inf-a-49dfaf-openshift-default-7967d4cccf Created pod: e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-a07920-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 33m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n Startup probe failed: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n Startup probe error: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused... 33m Normal Started pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Container started 33m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh Startup probe error: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh Startup probe failed: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 Startup probe error: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 Startup probe failed: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused 33m Normal AddedInterface pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Add eth0 [10.134.0.226/23] from ovn-kubernetes 33m Warning Unhealthy pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-h24nv Readiness probe error: HTTP probe failed with statuscode: 503... 33m Normal Pulled pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Container created 33m Normal Created pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Container created 33m Normal Started pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Container started 33m Warning ProbeError pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Startup probe error: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused... 33m Normal AddedInterface pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Add eth0 [10.132.0.168/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning Unhealthy pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Startup probe failed: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Readiness probe error: Get "http://10.134.0.216:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/e2e-inf-a-43c02d-openshift-default-57c4b85d79-5j69t Readiness probe failed: Get "http://10.134.0.216:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 Startup probe error: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 Startup probe failed: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b Startup probe failed: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b Startup probe error: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused... 33m Normal Killing pod/payload-processing-e2e-inf-b-2a8b26-5fc6db7f6d-25x6n Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz Stopping container payload-pre-processing 33m Warning ProbeError pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Startup probe error: Get "http://10.134.0.226:15021/healthz/ready": dial tcp 10.134.0.226:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Startup probe failed: Get "http://10.134.0.226:15021/healthz/ready": dial tcp 10.134.0.226:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Startup probe failed: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Startup probe error: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused... 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-a07920-6f78447f99-lzdrc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-2a8b26-857bc754ff-kvmnz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/payload-processing-e2e-inf-a-820c04-7b94544c4b-5sgbd Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd Stopping container payload-pre-processing 32m Normal Killing pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-mzh44 Stopping container istio-proxy 32m Normal Killing pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Stopping container istio-proxy 32m Warning Unhealthy pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Killing pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-pcb48 Stopping container istio-proxy 32m Warning ProbeError pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-m7x6z Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal Killing pod/e2e-inf-b-a07920-openshift-default-5f4bd748c-dvkv9 Stopping container istio-proxy 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-820c04-5cd68568bc-q44kd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Stopping container istio-proxy 32m Normal AddedLabel gateway/e2e-shared-a-w4-6c4f6f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-6c4f6f 32m Normal AddedInterface pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Add eth0 [10.134.0.230/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-6c4f6f-openshift-default Scaled up replica set e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57 from 0 to 1 32m Warning Unhealthy pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57 Created pod: e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 32m Warning ProbeError pod/e2e-inf-b-2a8b26-openshift-default-84b87df95f-j98mf Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c Error creating: pods "payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-6c4f6f: serviceaccount "payload-processing-e2e-shared-a-w4-6c4f6f" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986 Created pod: payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx 32m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986 Error creating: pods "payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-6c4f6f: serviceaccount "payload-processing-e2e-shared-a-w4-6c4f6f" not found 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-6c4f6f Scaled up replica set payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-6c4f6f Scaled up replica set payload-processing-e2e-shared-a-w4-6c4f6f-69b667986 from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c Created pod: payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s 32m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s Container created 32m Normal Created pod/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx Container created 32m Normal Started pod/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx Container started 32m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s Add eth0 [10.134.0.231/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s Container started 32m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Pulled pod/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx Add eth0 [10.134.0.232/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7bdc57c4cf from 6 to 5 32m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-dhlj6 Stopping container discovery 32m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx 32m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-dhlj6 32m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 32m Normal Killing pod/maas-default-gateway-openshift-default-7bdc57c4cf-zw4lx Stopping container istio-proxy 31m Normal Killing pod/e2e-inf-a-820c04-openshift-default-bc774f4f-k2sxq Stopping container istio-proxy 31m Normal Killing pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-inf-a-820c04-openshift-default-bc774f4f-t8k44 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Deleted pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 31m Normal ScalingReplicaSet deployment/e2e-worker-w3-b3593a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-b3593a-openshift-default-c5596b489 from 3 to 2 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-b3593a-openshift-default-c5596b489 Deleted pod: e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw 31m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-z7sb9 Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-b3593a-openshift-default New size: 2; reason: All metrics below target 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default New size: 3; reason: All metrics below target 31m Normal SuccessfulDelete replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Deleted pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp 31m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-hs9vp Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-2kkzw Stopping container istio-proxy 31m Normal Killing pod/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-inf-a-49dfaf-b8746c684-mpl56 Stopping container payload-processing 31m Normal Killing pod/payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl Stopping container payload-pre-processing 31m Warning Unhealthy pod/payload-processing-e2e-inf-b-dd257f-86d8696c74-cbqnr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-dd257f-6c8945d8d7-56lmb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-49dfaf-777485656c-7n8wn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-780077-8fb65bb75-dr7jl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-processing-e2e-inf-a-780077-667bdfc946-hdlwh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal Killing pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Stopping container istio-proxy 31m Normal Killing pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-lcrmj Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning ProbeError pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-inf-a-49dfaf-openshift-default-7967d4cccf-qlbg6 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal Killing pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-n7nvr Stopping container istio-proxy 31m Normal Killing pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Stopping container istio-proxy 31m Warning ProbeError pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-inf-b-dd257f-openshift-default-748b4d89df-kbn5k Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal Killing pod/payload-processing-e2e-inf-b-97ceef-684cfb864-gxprj Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s Stopping container payload-pre-processing 30m Normal Killing pod/e2e-inf-a-780077-openshift-default-7896f6cfc-w5chx Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Stopping container istio-proxy 30m Warning ProbeError pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-inf-a-780077-openshift-default-7896f6cfc-vsvvm Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal Started pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Container started 30m Warning Unhealthy pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Readiness probe failed: Get "http://10.134.0.230:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 30m Normal Pulled pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Container created 30m Warning ProbeError pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Readiness probe error: Get "http://10.134.0.230:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Container created 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Container started 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-33e759-openshift-default Scaled up replica set e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc from 0 to 1 30m Normal AddedLabel gateway/e2e-shared-b-w4-33e759 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-33e759 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc Created pod: e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb 30m Normal AddedInterface pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb Add eth0 [10.134.0.247/23] from ovn-kubernetes 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr Stopping container payload-processing 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd Stopping container payload-processing 30m Warning Unhealthy pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Startup probe failed: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Startup probe error: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9 Created pod: payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-33e759 Scaled up replica set payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4 from 0 to 1 30m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4 Error creating: pods "payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-33e759: serviceaccount "payload-processing-e2e-shared-b-w4-33e759" not found 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-33e759: serviceaccount "payload-processing-e2e-shared-b-w4-33e759" not found 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-97ceef-86cf87f57f-9m55s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-33e759 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9 from 0 to 1 30m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v Container started 30m Normal Started pod/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 Container started 30m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Created pod/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 Container created 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4 Created pod: payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 30m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v Container created 30m Normal Pulled pod/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 Add eth0 [10.134.0.249/23] from ovn-kubernetes 30m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v Add eth0 [10.134.0.248/23] from ovn-kubernetes 30m Normal AddedInterface pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms Add eth0 [10.132.0.169/23] from ovn-kubernetes 30m Normal SuccessfulCreate replicaset/e2e-inf-b-97ceef-openshift-default-77d68d9ff9 Created pod: e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-97ceef-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-inf-b-97ceef-openshift-default Scaled up replica set e2e-inf-b-97ceef-openshift-default-77d68d9ff9 from 1 to 2 30m Normal Created pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms Container created 30m Normal Pulled pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms Container started 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-6c4f6f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-6c4f6f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f Created pod: e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q 30m Normal AddedInterface pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q Add eth0 [10.134.0.251/23] from ovn-kubernetes 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-969b92-openshift-default Scaled up replica set e2e-shared-a-w2-969b92-openshift-default-b496c6b6f from 4 to 5 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-969b92-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 30m Normal AddedInterface pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv Add eth0 [10.133.0.153/23] from ovn-kubernetes 30m Normal Started pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q Container started 30m Normal ScalingReplicaSet deployment/e2e-inf-b-1da7b2-openshift-default Scaled up replica set e2e-inf-b-1da7b2-openshift-default-6997bcc7b7 from 2 to 3 30m Normal SuccessfulCreate replicaset/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7 Created pod: e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-1da7b2-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 30m Normal Created pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q Container created 30m Normal Pulled pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Pulled pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv Container started 30m Normal Created pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv Container created 30m Warning Unhealthy pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms Startup probe failed: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms Startup probe error: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q Startup probe failed: Get "http://10.134.0.251:15021/healthz/ready": dial tcp 10.134.0.251:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q Startup probe error: Get "http://10.134.0.251:15021/healthz/ready": dial tcp 10.134.0.251:15021: connect: connection refused... 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-16c1bb-openshift-default Scaled up replica set e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 from 1 to 4 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h Add eth0 [10.133.0.154/23] from ovn-kubernetes 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb Container started 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h Container started 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb Container created 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb Add eth0 [10.134.0.252/23] from ovn-kubernetes 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq Container started 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq Container created 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq Add eth0 [10.134.0.253/23] from ovn-kubernetes 30m Warning Unhealthy pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv Startup probe failed: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h Container created 30m Warning ProbeError pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv Startup probe error: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused... 30m Normal Killing pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-dq5ms Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Stopping container istio-proxy 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-d01638-openshift-default-884b45857 Created pod: e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx 30m Normal AddedLabel gateway/e2e-shared-a-w5-d01638 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-d01638 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-d01638-openshift-default Scaled up replica set e2e-shared-a-w5-d01638-openshift-default-884b45857 from 0 to 1 30m Normal AddedInterface pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Add eth0 [10.132.0.171/23] from ovn-kubernetes 30m Normal Killing pod/e2e-inf-a-cf3466-openshift-default-97cd47b87-cls2c Stopping container istio-proxy 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-6c4f6f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-1da7b2-74c4ddb675-hbkbt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-6c4f6f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning Unhealthy pod/payload-processing-e2e-inf-b-1da7b2-6cbbfdc77b-nc6fd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb Startup probe failed: Get "http://10.134.0.252:15021/healthz/ready": dial tcp 10.134.0.252:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb Startup probe error: Get "http://10.134.0.252:15021/healthz/ready": dial tcp 10.134.0.252:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq Startup probe error: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq Startup probe failed: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning Unhealthy pod/payload-processing-e2e-inf-a-cf3466-5858df687b-khkpr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning ProbeError pod/e2e-inf-b-97ceef-openshift-default-77d68d9ff9-2rqf5 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-cf3466-56cf7bd867-xltzf Readiness probe failed: timeout: failed to connect service "10.134.0.209:50052" within 1s: context deadline exceeded 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-d01638: serviceaccount "payload-processing-e2e-shared-a-w5-d01638" not found 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787 Created pod: payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-j758b 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-d01638 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787 from 0 to 1 30m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-d01638-5668596858 Error creating: pods "payload-processing-e2e-shared-a-w5-d01638-5668596858-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-d01638: serviceaccount "payload-processing-e2e-shared-a-w5-d01638" not found 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-d01638-5668596858 Created pod: payload-processing-e2e-shared-a-w5-d01638-5668596858-chl8d 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-d01638 Scaled up replica set payload-processing-e2e-shared-a-w5-d01638-5668596858 from 0 to 1 30m Normal Pulled pod/payload-processing-e2e-shared-a-w5-d01638-5668596858-chl8d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-j758b Container started 30m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-j758b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-d01638-5668596858-chl8d Add eth0 [10.134.0.255/23] from ovn-kubernetes 30m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-j758b Add eth0 [10.132.0.172/23] from ovn-kubernetes 30m Normal Created pod/payload-processing-e2e-shared-a-w5-d01638-5668596858-chl8d Container created 30m Normal Started pod/payload-processing-e2e-shared-a-w5-d01638-5668596858-chl8d Container started 30m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-j758b Container created 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default New size: 8; reason: 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-16c1bb-openshift-default Scaled up replica set e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 from 4 to 8 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 Add eth0 [10.134.1.1/23] from ovn-kubernetes 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t Add eth0 [10.132.0.174/23] from ovn-kubernetes 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft Container started 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn Add eth0 [10.132.0.173/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t Container started 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft Add eth0 [10.133.0.155/23] from ovn-kubernetes 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn Container created 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn Container started 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft Container created 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t Container created 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 Container created 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 Container started 30m Normal Killing pod/payload-processing-e2e-inf-b-f96ede-55bbbb7db8-956nk Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r Stopping container payload-pre-processing 30m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 Startup probe error: Get "http://10.134.1.1:15021/healthz/ready": dial tcp 10.134.1.1:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn Startup probe failed: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused 30m Normal Killing pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-mflx8 Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 Startup probe failed: Get "http://10.134.1.1:15021/healthz/ready": dial tcp 10.134.1.1:15021: connect: connection refused 30m Normal Killing pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Stopping container istio-proxy 30m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn Startup probe error: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t Startup probe failed: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t Startup probe error: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused... 30m Normal Killing pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-5z9dv Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft Startup probe failed: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft Startup probe error: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-inf-b-1da7b2-openshift-default-6997bcc7b7-ch8zj Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-16c1bb-openshift-default Scaled up replica set e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 from 8 to 10 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default New size: 10; reason: 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Created pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx Container started 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx Container created 30m Normal Started pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n Container started 30m Normal Created pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n Container created 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n Add eth0 [10.133.0.156/23] from ovn-kubernetes 30m Normal AddedInterface pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx Add eth0 [10.132.0.175/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n Startup probe failed: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n Startup probe error: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused... 29m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx Startup probe error: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx Startup probe failed: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-f96ede-9b55fbd79-sph6r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-bea3ef-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-bea3ef-openshift-default Scaled up replica set e2e-shared-a-w0-bea3ef-openshift-default-784b4db987 from 1 to 2 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987 Created pod: e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 29m Normal Started pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 Container started 29m Normal Created pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 Container created 29m Normal Pulled pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 Add eth0 [10.132.0.176/23] from ovn-kubernetes 29m Warning FailedUpdateStatus horizontalpodautoscaler/e2e-inf-b-f96ede-openshift-default horizontalpodautoscalers.autoscaling "e2e-inf-b-f96ede-openshift-default" not found 29m Normal Killing pod/e2e-inf-b-f96ede-openshift-default-684654576c-bnnd7 Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-b-f96ede-openshift-default-684654576c-whs76 Stopping container istio-proxy 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 29m Normal AddedLabel gateway/e2e-shared-a-w3-320045 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-320045 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-320045-openshift-default Scaled up replica set e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c from 0 to 1 29m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 Add eth0 [10.134.1.5/23] from ovn-kubernetes 29m Warning ProbeError pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 Startup probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 Startup probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-320045-8557b8498 Created pod: payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-320045: serviceaccount "payload-processing-e2e-shared-a-w3-320045" not found 29m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-320045-8557b8498 Error creating: pods "payload-processing-e2e-shared-a-w3-320045-8557b8498-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-320045: serviceaccount "payload-processing-e2e-shared-a-w3-320045" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99 Created pod: payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-320045 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-320045 Scaled up replica set payload-processing-e2e-shared-a-w3-320045-8557b8498 from 0 to 1 29m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 Add eth0 [10.134.1.8/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p Add eth0 [10.134.1.7/23] from ovn-kubernetes 29m Normal Started pod/payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 Container started 29m Normal Pulled pod/payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p Container started 29m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p Container created 29m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 Container created 29m Warning Unhealthy pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Readiness probe failed: Get "http://10.132.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning ProbeError pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Readiness probe error: Get "http://10.132.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Normal AddedLabel gateway/e2e-shared-b-w5-8e24a5 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-8e24a5 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-8e24a5-openshift-default Scaled up replica set e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c from 0 to 1 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c Created pod: e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 29m Normal Pulled pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 Container created 29m Normal AddedInterface pod/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 Add eth0 [10.134.1.10/23] from ovn-kubernetes 29m Normal Started pod/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 Container started 29m Normal Pulled pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Pulled pod/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Container created 29m Normal Started pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Container started 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-33e759-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal Started pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb Container started 29m Warning Unhealthy pod/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 Startup probe failed: Get "http://10.134.1.10:15021/healthz/ready": dial tcp 10.134.1.10:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 Startup probe error: Get "http://10.134.1.10:15021/healthz/ready": dial tcp 10.134.1.10:15021: connect: connection refused... 29m Normal Created pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb Container created 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-33e759-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d Error creating: pods "payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-8e24a5: serviceaccount "payload-processing-e2e-shared-b-w5-8e24a5" not found 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f Error creating: pods "payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-8e24a5: serviceaccount "payload-processing-e2e-shared-b-w5-8e24a5" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d Created pod: payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-8e24a5 Scaled up replica set payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d from 0 to 1 29m Warning Unhealthy pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f Created pod: payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 29m Warning ProbeError pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-8e24a5 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f from 0 to 1 28m Warning Unhealthy pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb Startup probe failed: Get "http://10.134.0.247:15021/healthz/ready": dial tcp 10.134.0.247:15021: connect: connection refused 28m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff Add eth0 [10.132.0.178/23] from ovn-kubernetes 28m Normal Pulled pod/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 Container created 28m Normal Created pod/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff Container created 28m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 Container started 28m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 Add eth0 [10.134.1.13/23] from ovn-kubernetes 28m Warning ProbeError pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb Startup probe error: Get "http://10.134.0.247:15021/healthz/ready": dial tcp 10.134.0.247:15021: connect: connection refused... 28m Normal Started pod/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff Container started 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-d01638-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-d01638-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-8e24a5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-8e24a5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-d01638-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-d01638-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-62f5d1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585 Created pod: e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-62f5d1-openshift-default Scaled up replica set e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585 from 1 to 2 28m Normal AddedInterface pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Add eth0 [10.132.0.180/23] from ovn-kubernetes 28m Normal Started pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Container started 28m Normal Created pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Container created 28m Normal Pulled pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-8e24a5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-8e24a5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-33e759-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-33e759-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning ProbeError pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Startup probe error: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Startup probe failed: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused 28m Normal AddedLabel gateway/e2e-shared-b-w3-f69104 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-f69104 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-f69104-openshift-default Scaled up replica set e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7 from 0 to 1 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7 Created pod: e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz 28m Normal AddedInterface pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Add eth0 [10.134.1.15/23] from ovn-kubernetes 28m Normal SuccessfulDelete replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Deleted pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w 28m Normal SuccessfulDelete replicaset/e2e-worker-w6-5062f6-openshift-default-68b95c9cff Deleted pod: e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5062f6-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 28m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Stopping container istio-proxy 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default New size: 3; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w6-5062f6-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-5062f6-openshift-default-68b95c9cff from 3 to 2 28m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jqv4p Stopping container istio-proxy 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-f69104 Scaled up replica set payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb from 0 to 1 28m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-99w7w Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb Error creating: pods "payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-f69104: serviceaccount "payload-processing-e2e-shared-b-w3-f69104" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-f69104 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f Error creating: pods "payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-f69104: serviceaccount "payload-processing-e2e-shared-b-w3-f69104" not found 28m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf Add eth0 [10.133.0.157/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb Created pod: payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj 28m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj Add eth0 [10.133.0.158/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f Created pod: payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf 28m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf Container started 28m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf Container created 28m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Pulled pod/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj Container started 28m Normal Created pod/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj Container created 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-320045-openshift-default Scaled up replica set e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c from 1 to 4 28m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 Container created 28m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn Add eth0 [10.132.0.181/23] from ovn-kubernetes 28m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 Container started 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-320045-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs 28m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b Add eth0 [10.133.0.160/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b 28m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs Add eth0 [10.134.1.16/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn Container started 28m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs Container started 28m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn Container created 28m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b Container started 28m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b Container created 28m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs Container created 28m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b Startup probe error: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs Startup probe failed: Get "http://10.134.1.16:15021/healthz/ready": dial tcp 10.134.1.16:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 Startup probe error: Get "http://10.134.1.5:15021/healthz/ready": dial tcp 10.134.1.5:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn Startup probe failed: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn Startup probe error: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused... 28m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs Startup probe error: Get "http://10.134.1.16:15021/healthz/ready": dial tcp 10.134.1.16:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b Startup probe failed: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused 28m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 Startup probe failed: Get "http://10.134.1.5:15021/healthz/ready": dial tcp 10.134.1.5:15021: connect: connection refused 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-320045-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-320045-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57 Created pod: e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-6c4f6f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-6c4f6f-openshift-default Scaled up replica set e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57 from 1 to 2 27m Normal AddedInterface pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc Add eth0 [10.132.0.182/23] from ovn-kubernetes 27m Normal Started pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc Container started 27m Normal Created pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc Container created 27m Normal Pulled pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Warning Unhealthy pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc Startup probe failed: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc Startup probe error: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused... 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-320045-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-320045-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-320045-openshift-default New size: 7; reason: All metrics below target 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-320045-openshift-default Scaled up replica set e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c from 4 to 7 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs 27m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr Container started 27m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs Add eth0 [10.132.0.183/23] from ovn-kubernetes 27m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr Container created 27m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l Add eth0 [10.134.1.20/23] from ovn-kubernetes 27m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs Container started 27m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs Container created 27m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr Add eth0 [10.133.0.164/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l Container started 27m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l Container created 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-62f5d1-openshift-default Scaled up replica set e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585 from 2 to 3 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585 Created pod: e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-62f5d1-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 27m Normal Pulled pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm Container created 27m Normal Started pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm Container started 27m Normal AddedInterface pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm Add eth0 [10.133.0.166/23] from ovn-kubernetes 27m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l Startup probe error: Get "http://10.134.1.20:15021/healthz/ready": dial tcp 10.134.1.20:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs Startup probe failed: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l Startup probe failed: Get "http://10.134.1.20:15021/healthz/ready": dial tcp 10.134.1.20:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs Startup probe error: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr Startup probe failed: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr Startup probe error: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm Startup probe failed: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm Startup probe error: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused... 27m Normal Started pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Container started 27m Normal Pulled pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Warning ProbeError pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Readiness probe error: Get "http://10.134.1.15:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-33e759-openshift-default Scaled up replica set e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc from 1 to 2 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-33e759-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc Created pod: e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv 27m Warning Unhealthy pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Readiness probe failed: Get "http://10.134.1.15:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 27m Normal AddedInterface pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Add eth0 [10.132.0.185/23] from ovn-kubernetes 27m Normal Created pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Container created 27m Normal AddedLabel gateway/e2e-ipp-a-f91e87 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f91e87 27m Normal ScalingReplicaSet deployment/e2e-ipp-a-f91e87-openshift-default Scaled up replica set e2e-ipp-a-f91e87-openshift-default-6d7b498dff from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-f91e87-openshift-default-6d7b498dff Created pod: e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m 27m Normal AddedInterface pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Add eth0 [10.134.1.21/23] from ovn-kubernetes 26m Warning ProbeError pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd Created pod: payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-qzn6j 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f91e87 Scaled up replica set payload-processing-e2e-ipp-a-f91e87-cc47bfbdd from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f91e87 Scaled up replica set payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd Error creating: pods "payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f91e87: serviceaccount "payload-processing-e2e-ipp-a-f91e87" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47 Created pod: payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-pzzpg 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47 Error creating: pods "payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f91e87: serviceaccount "payload-processing-e2e-ipp-a-f91e87" not found 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-pzzpg Add eth0 [10.132.0.186/23] from ovn-kubernetes 26m Normal Started pod/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-qzn6j Container started 26m Normal Created pod/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-qzn6j Container created 26m Normal Pulled pod/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-qzn6j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-qzn6j Add eth0 [10.134.1.22/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-pzzpg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-pzzpg Container created 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-pzzpg Container started 26m Normal Pulled pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Container created 26m Normal Started pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Container started 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c Created pod: e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-60e8f4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-60e8f4-openshift-default Scaled up replica set e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c from 1 to 2 26m Normal Started pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc Container started 26m Warning ProbeError pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Startup probe error: Get "http://10.134.1.15:15021/healthz/ready": dial tcp 10.134.1.15:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Startup probe failed: Get "http://10.134.1.15:15021/healthz/ready": dial tcp 10.134.1.15:15021: connect: connection refused 26m Normal AddedInterface pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc Add eth0 [10.134.1.23/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc Container created 26m Warning Unhealthy pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc Startup probe failed: Get "http://10.134.1.23:15021/healthz/ready": dial tcp 10.134.1.23:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc Startup probe error: Get "http://10.134.1.23:15021/healthz/ready": dial tcp 10.134.1.23:15021: connect: connection refused... 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-f69104-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-f69104-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-481249-openshift-default-6c77499cf5 Created pod: e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-481249-openshift-default Scaled up replica set e2e-ipp-a-481249-openshift-default-6c77499cf5 from 0 to 1 26m Normal AddedLabel gateway/e2e-ipp-a-481249 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-481249 26m Normal AddedInterface pod/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 Add eth0 [10.134.1.25/23] from ovn-kubernetes 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-f69104-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-f69104-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5 Error creating: pods "payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-481249: serviceaccount "payload-processing-e2e-ipp-a-481249" not found 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-481249 Scaled up replica set payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-481249 Scaled up replica set payload-processing-e2e-ipp-a-481249-65677df64 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5 Created pod: payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-jqjkm 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-481249-65677df64 Created pod: payload-processing-e2e-ipp-a-481249-65677df64-8bpkm 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-481249-65677df64 Error creating: pods "payload-processing-e2e-ipp-a-481249-65677df64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-481249: serviceaccount "payload-processing-e2e-ipp-a-481249" not found 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-jqjkm Container started 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-jqjkm Container created 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-jqjkm Add eth0 [10.133.0.174/23] from ovn-kubernetes 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-481249-65677df64-8bpkm Add eth0 [10.133.0.173/23] from ovn-kubernetes 26m Normal Pulled pod/payload-processing-e2e-ipp-a-481249-65677df64-8bpkm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-processing-e2e-ipp-a-481249-65677df64-8bpkm Container created 26m Normal Started pod/payload-processing-e2e-ipp-a-481249-65677df64-8bpkm Container started 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-jqjkm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 26m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-wvbtj 26m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-wvbtj Add eth0 [10.134.1.27/23] from ovn-kubernetes 25m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-wvbtj Container started 25m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-wvbtj Container created 25m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-wvbtj Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7 Created pod: e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-ea2547-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-ea2547-openshift-default Scaled up replica set e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7 from 1 to 2 25m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-wvbtj Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal AddedInterface pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Add eth0 [10.133.0.178/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Container started 25m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-wvbtj Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal Created pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Container created 25m Warning Unhealthy pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Startup probe failed: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Startup probe error: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused... 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-abf8ae-openshift-default-578749ddbf Created pod: e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 25m Normal AddedLabel gateway/e2e-ipp-a-abf8ae Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-abf8ae 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-abf8ae-openshift-default Scaled up replica set e2e-ipp-a-abf8ae-openshift-default-578749ddbf from 0 to 1 25m Normal Pulled pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Add eth0 [10.132.0.188/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc Created pod: e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-33e759-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-33e759-openshift-default Scaled up replica set e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc from 2 to 3 25m Normal AddedInterface pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Add eth0 [10.133.0.179/23] from ovn-kubernetes 25m Normal Created pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Container created 25m Normal Started pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Container started 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7bdc57c4cf from 5 to 4 25m Normal Killing pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-abf8ae Scaled up replica set payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8 Created pod: payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-fslvf 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7 Created pod: payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-9h869 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7 Error creating: pods "payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-abf8ae: serviceaccount "payload-processing-e2e-ipp-a-abf8ae" not found 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-abf8ae Scaled up replica set payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8 from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8 Error creating: pods "payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-abf8ae: serviceaccount "payload-processing-e2e-ipp-a-abf8ae" not found 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-9h869 Container started 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-9h869 Add eth0 [10.133.0.180/23] from ovn-kubernetes 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-9h869 Container created 25m Normal Started pod/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-fslvf Container started 25m Normal Created pod/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-fslvf Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-fslvf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-fslvf Add eth0 [10.133.0.181/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-9h869 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-mkm92 Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-d1ae36-openshift-default Scaled up replica set e2e-ipp-a-d1ae36-openshift-default-56d5b87b55 from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55 Created pod: e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm 25m Normal AddedLabel gateway/e2e-ipp-a-d1ae36 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-d1ae36 25m Normal AddedInterface pod/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm Add eth0 [10.132.0.193/23] from ovn-kubernetes 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 25m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-worker-w2-40ac41-openshift-default-5c6c78d655 Deleted pod: e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr 25m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w Stopping container istio-proxy 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-40ac41-openshift-default New size: 2; reason: All metrics below target 25m Normal ScalingReplicaSet deployment/e2e-worker-w2-40ac41-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-40ac41-openshift-default-5c6c78d655 from 3 to 2 25m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 3 to 2 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44 Error creating: pods "payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d1ae36: serviceaccount "payload-processing-e2e-ipp-a-d1ae36" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-d1ae36 Scaled up replica set payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9 from 0 to 1 25m Normal SuccessfulDelete replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Deleted pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-qpq5w 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-d1ae36 Scaled up replica set payload-processing-e2e-ipp-a-d1ae36-65d75fbc44 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9 Error creating: pods "payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d1ae36: serviceaccount "payload-processing-e2e-ipp-a-d1ae36" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44 Created pod: payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-75bdf 25m Normal Pulled pod/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-75bdf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-worker-w1-beb8d5-openshift-default-fc65648b7 Deleted pod: e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr 25m Normal ScalingReplicaSet deployment/e2e-worker-w1-beb8d5-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-beb8d5-openshift-default-fc65648b7 from 3 to 2 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-beb8d5-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6add35-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 25m Normal ScalingReplicaSet deployment/e2e-worker-w4-6add35-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-6add35-openshift-default-9c5d9d5ff from 3 to 2 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9 Created pod: payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-5tsn5 25m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff Deleted pod: e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-75bdf Add eth0 [10.132.0.194/23] from ovn-kubernetes 25m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-5tsn5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-75bdf Container created 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-5tsn5 Add eth0 [10.133.0.193/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-5tsn5 Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-5tsn5 Container created 25m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-xgdhr Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal Started pod/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-75bdf Container started 25m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-tlz7l Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-rpfsr Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal Killing pod/payload-processing-e2e-ipp-a-f91e87-cc47bfbdd-qzn6j Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f91e87-6759f47c47-pzzpg Stopping container payload-pre-processing 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-f91e87-openshift-default-6d7b498dff Created pod: e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-f91e87-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-f91e87-openshift-default Scaled up replica set e2e-ipp-a-f91e87-openshift-default-6d7b498dff from 1 to 2 25m Normal Started pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z Container started 25m Normal Created pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z Container created 25m Normal AddedInterface pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z Add eth0 [10.132.0.196/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Readiness probe failed: Get "http://10.134.1.21:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Warning ProbeError pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Readiness probe error: Get "http://10.134.1.21:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Warning Unhealthy pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z Startup probe failed: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z Startup probe error: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused... 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-60e8f4-openshift-default Scaled up replica set e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c from 2 to 4 25m Normal Pulled pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Container created 25m Normal Started pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Container started 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c Created pod: e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c Created pod: e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-60e8f4-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal Pulled pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z Add eth0 [10.132.0.197/23] from ovn-kubernetes 25m Normal AddedInterface pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 Add eth0 [10.134.1.37/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z Container created 25m Normal Started pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z Container started 25m Normal Started pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 Container started 25m Normal Created pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 Container created 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-320045-openshift-default Scaled up replica set e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c from 7 to 10 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-320045-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c (combined from similar events): Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c Created pod: e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 24m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg Add eth0 [10.134.1.38/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb Container started 24m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb Container created 24m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb Add eth0 [10.133.0.195/23] from ovn-kubernetes 24m Normal AddedInterface pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 Add eth0 [10.133.0.194/23] from ovn-kubernetes 24m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg Container created 24m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg Container started 24m Normal Started pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 Container started 24m Normal Created pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 Container created 24m Normal Pulled pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Warning ProbeError pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Startup probe error: Get "http://10.134.1.21:15021/healthz/ready": dial tcp 10.134.1.21:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Startup probe failed: Get "http://10.134.1.21:15021/healthz/ready": dial tcp 10.134.1.21:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z Startup probe error: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z Startup probe failed: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 Startup probe error: Get "http://10.134.1.37:15021/healthz/ready": dial tcp 10.134.1.37:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 Startup probe failed: Get "http://10.134.1.37:15021/healthz/ready": dial tcp 10.134.1.37:15021: connect: connection refused 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-481249-5dfdbbf4f5-jqjkm Stopping container payload-pre-processing 24m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg Startup probe error: Get "http://10.134.1.38:15021/healthz/ready": dial tcp 10.134.1.38:15021: connect: connection refused... 24m Normal Killing pod/payload-processing-e2e-ipp-a-481249-65677df64-8bpkm Stopping container payload-processing 24m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg Startup probe failed: Get "http://10.134.1.38:15021/healthz/ready": dial tcp 10.134.1.38:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 Startup probe error: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused... 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-89a676-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 24m Normal ScalingReplicaSet deployment/e2e-worker-w5-89a676-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-89a676-openshift-default-6bdc5cc785 from 3 to 2 24m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 Startup probe failed: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused 24m Warning Unhealthy pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused 24m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/e2e-worker-w5-89a676-openshift-default-6bdc5cc785 Deleted pod: e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj 24m Warning ProbeError pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kjkfj Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6 Error creating: pods "payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f91e87: serviceaccount "payload-processing-e2e-ipp-a-f91e87" not found 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9 Error creating: pods "payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f91e87: serviceaccount "payload-processing-e2e-ipp-a-f91e87" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f91e87 Scaled up replica set payload-processing-e2e-ipp-a-f91e87-7bf7b54d6 from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f91e87 Scaled up replica set payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6 Created pod: payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9 Created pod: payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 Add eth0 [10.132.0.201/23] from ovn-kubernetes 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w Add eth0 [10.132.0.200/23] from ovn-kubernetes 24m Normal Pulling pod/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 24m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 24m Normal Pulled pod/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.936s (1.936s including waiting). Image size: 166612641 bytes. 24m Normal Created pod/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 Container created 24m Normal Started pod/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 Container started 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w Container started 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.981s (1.981s including waiting). Image size: 166612641 bytes. 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w Container created 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-969b92-openshift-default Scaled down replica set e2e-shared-a-w2-969b92-openshift-default-b496c6b6f from 5 to 4 24m Normal SuccessfulDelete replicaset/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f Deleted pod: e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-969b92-openshift-default New size: 4; reason: All metrics below target 24m Normal Killing pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-fc45q Stopping container istio-proxy 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f91e87-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f91e87-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a42e5-openshift-default-84779c8749 Created pod: e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 24m Warning Unhealthy pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Normal AddedLabel gateway/e2e-ipp-b-1a42e5 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-1a42e5 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-1a42e5-openshift-default Scaled up replica set e2e-ipp-b-1a42e5-openshift-default-84779c8749 from 0 to 1 24m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 Readiness probe failed: timeout: failed to connect service "10.132.0.118:50052" within 1s: context deadline exceeded 24m Normal AddedInterface pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Add eth0 [10.133.0.196/23] from ovn-kubernetes 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1a42e5-8b4998c78 Error creating: pods "payload-processing-e2e-ipp-b-1a42e5-8b4998c78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a42e5: serviceaccount "payload-processing-e2e-ipp-b-1a42e5" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1a42e5 Scaled up replica set payload-processing-e2e-ipp-b-1a42e5-8b4998c78 from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1a42e5 Scaled up replica set payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9 Created pod: payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-bpmp7 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9 Error creating: pods "payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a42e5: serviceaccount "payload-processing-e2e-ipp-b-1a42e5" not found 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-bpmp7 Add eth0 [10.133.0.198/23] from ovn-kubernetes 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-bpmp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1a42e5-8b4998c78 Created pod: payload-processing-e2e-ipp-b-1a42e5-8b4998c78-p9lfr 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-bpmp7 Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-bpmp7 Container created 24m Normal Created pod/payload-processing-e2e-ipp-b-1a42e5-8b4998c78-p9lfr Container created 24m Normal Started pod/payload-processing-e2e-ipp-b-1a42e5-8b4998c78-p9lfr Container started 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1a42e5-8b4998c78-p9lfr Add eth0 [10.134.1.40/23] from ovn-kubernetes 24m Normal Pulled pod/payload-processing-e2e-ipp-b-1a42e5-8b4998c78-p9lfr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Warning Unhealthy pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Readiness probe failed: Get "http://10.132.0.188:15021/healthz/ready": read tcp 10.132.0.2:34876->10.132.0.188:15021: read: connection reset by peer 24m Warning ProbeError pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Readiness probe error: Get "http://10.132.0.188:15021/healthz/ready": read tcp 10.132.0.2:34876->10.132.0.188:15021: read: connection reset by peer... 24m Normal Pulled pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Container started 24m Normal Created pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Container created 24m Normal Pulled pod/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 Container created 24m Normal Started pod/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 Container started 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-481249-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-481249-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning ProbeError pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 24m Warning Unhealthy pod/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 Startup probe failed: Get "http://10.134.1.25:15021/healthz/ready": dial tcp 10.134.1.25:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 Startup probe error: Get "http://10.134.1.25:15021/healthz/ready": dial tcp 10.134.1.25:15021: connect: connection refused... 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-f69104-openshift-default Scaled up replica set e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7 from 1 to 2 24m Normal AddedInterface pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 Add eth0 [10.133.0.200/23] from ovn-kubernetes 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7 Created pod: e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 24m Normal Pulled pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 Container created 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-f69104-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 24m Normal Started pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 Container started 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-abf8ae-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-abf8ae-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-abf8ae-7f58d987b7-9h869 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-a-abf8ae-5b7f5777f8-fslvf Stopping container payload-processing 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9 Created pod: e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-95492e-openshift-default Scaled up replica set e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9 from 1 to 2 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-95492e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 24m Normal Created pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q Container created 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-481249 Scaled up replica set payload-pre-processing-e2e-ipp-a-481249-6686b74599 from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-481249 Scaled up replica set payload-processing-e2e-ipp-a-481249-6d59bcdc6b from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-481249-6d59bcdc6b Error creating: pods "payload-processing-e2e-ipp-a-481249-6d59bcdc6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-481249: serviceaccount "payload-processing-e2e-ipp-a-481249" not found 24m Normal AddedInterface pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q Add eth0 [10.134.1.41/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-481249-6686b74599 Error creating: pods "payload-pre-processing-e2e-ipp-a-481249-6686b74599-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-481249: serviceaccount "payload-processing-e2e-ipp-a-481249" not found 24m Normal Started pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q Container started 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-481249-6d59bcdc6b Created pod: payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-481249-6686b74599 Created pod: payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l Add eth0 [10.134.1.42/23] from ovn-kubernetes 23m Normal Pulling pod/payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m Add eth0 [10.134.1.43/23] from ovn-kubernetes 23m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 23m Warning Unhealthy pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused... 23m Normal Created pod/payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l Container created 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1a42e5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1a42e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Pulled pod/payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.729s (2.729s including waiting). Image size: 166612641 bytes. 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.732s (2.732s including waiting). Image size: 166612641 bytes. 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m Container created 23m Warning ProbeError pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q Startup probe error: Get "http://10.134.1.41:15021/healthz/ready": dial tcp 10.134.1.41:15021: connect: connection refused... 23m Normal Started pod/payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l Container started 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m Container started 23m Warning Unhealthy pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q Startup probe failed: Get "http://10.134.1.41:15021/healthz/ready": dial tcp 10.134.1.41:15021: connect: connection refused 23m Normal Killing pod/payload-processing-e2e-ipp-a-d1ae36-65d75fbc44-75bdf Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-d1ae36-57964568c9-5tsn5 Stopping container payload-pre-processing 23m Normal Pulled pod/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm Container started 23m Normal Created pod/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm Container created 23m Normal AddedLabel gateway/e2e-ipp-b-38b47d Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-38b47d 23m Normal SuccessfulCreate replicaset/e2e-ipp-b-38b47d-openshift-default-649dd76fb5 Created pod: e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz 23m Normal ScalingReplicaSet deployment/e2e-ipp-b-38b47d-openshift-default Scaled up replica set e2e-ipp-b-38b47d-openshift-default-649dd76fb5 from 0 to 1 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-481249-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal AddedInterface pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Add eth0 [10.133.0.203/23] from ovn-kubernetes 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-481249-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-d1ae36-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-d1ae36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning ProbeError pod/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm Startup probe error: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm Startup probe failed: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-38b47d Scaled up replica set payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-38b47d Scaled up replica set payload-processing-e2e-ipp-b-38b47d-5d787c7c54 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-38b47d-5d787c7c54 Created pod: payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9 Error creating: pods "payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-38b47d: serviceaccount "payload-processing-e2e-ipp-b-38b47d" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9 Created pod: payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-38b47d-5d787c7c54 Error creating: pods "payload-processing-e2e-ipp-b-38b47d-5d787c7c54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-38b47d: serviceaccount "payload-processing-e2e-ipp-b-38b47d" not found 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c Add eth0 [10.133.0.204/23] from ovn-kubernetes 23m Normal Started pod/payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c Container started 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls Add eth0 [10.132.0.203/23] from ovn-kubernetes 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls Container started 23m Normal Created pod/payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c Container created 23m Normal Pulled pod/payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-abf8ae-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-abf8ae-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal AddedLabel gateway/e2e-ipp-a-5f99e7 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-5f99e7 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9 Created pod: e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m 23m Normal ScalingReplicaSet deployment/e2e-ipp-a-5f99e7-openshift-default Scaled up replica set e2e-ipp-a-5f99e7-openshift-default-65d869d4b9 from 0 to 1 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6 Error creating: pods "payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-abf8ae: serviceaccount "payload-processing-e2e-ipp-a-abf8ae" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895 Created pod: payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-abf8ae Scaled up replica set payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-abf8ae Scaled up replica set payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895 from 0 to 1 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895 Error creating: pods "payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-abf8ae: serviceaccount "payload-processing-e2e-ipp-a-abf8ae" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6 Created pod: payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b 23m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 Add eth0 [10.133.0.205/23] from ovn-kubernetes 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 Container started 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.305s (1.305s including waiting). Image size: 166612641 bytes. 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 Container created 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-d1ae36-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-d1ae36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-d1ae36 Scaled up replica set payload-processing-e2e-ipp-a-d1ae36-d65d966b6 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-d1ae36-d65d966b6 Created pod: payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-d1ae36 Scaled up replica set payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f Created pod: payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-d1ae36-d65d966b6 Error creating: pods "payload-processing-e2e-ipp-a-d1ae36-d65d966b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d1ae36: serviceaccount "payload-processing-e2e-ipp-a-d1ae36" not found 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f Error creating: pods "payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d1ae36: serviceaccount "payload-processing-e2e-ipp-a-d1ae36" not found 22m Warning ProbeError pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Readiness probe error: Get "http://10.133.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning Unhealthy pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Readiness probe failed: Get "http://10.133.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Normal Created pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Container created 22m Normal Started pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Container started 22m Normal Pulled pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Warning ProbeError pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Startup probe error: Get "http://10.133.0.196:15021/healthz/ready": dial tcp 10.133.0.196:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Startup probe failed: Get "http://10.133.0.196:15021/healthz/ready": dial tcp 10.133.0.196:15021: connect: connection refused 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-f69104-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-f69104-openshift-default Scaled up replica set e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7 from 2 to 3 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7 Created pod: e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 Stopping container payload-processing 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1a42e5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1a42e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-320045-8557b8498-ttln4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-320045-65b9b58b99-z8z7p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Started pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Container started 21m Normal Pulled pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Container created 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-38b47d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-38b47d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf Add eth0 [10.134.1.44/23] from ovn-kubernetes 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf Container created 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf Container started 21m Warning Unhealthy pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Startup probe failed: Get "http://10.133.0.203:15021/healthz/ready": dial tcp 10.133.0.203:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Startup probe error: Get "http://10.133.0.203:15021/healthz/ready": dial tcp 10.133.0.203:15021: connect: connection refused... 21m Normal AddedInterface pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss Add eth0 [10.134.1.45/23] from ovn-kubernetes 21m Normal Pulled pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss Container created 21m Normal Started pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss Container started 21m Warning ProbeError pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss Startup probe error: Get "http://10.134.1.45:15021/healthz/ready": dial tcp 10.134.1.45:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss Startup probe failed: Get "http://10.134.1.45:15021/healthz/ready": dial tcp 10.134.1.45:15021: connect: connection refused 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-57fgs Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-qmr9l Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-jphbb Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-4m7f7 Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-g4kjg Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-58ghs Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-kjtfr Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-bjxp7 Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-vgh7b Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-320045-openshift-default-6dcdf94b5c-n4xkn Stopping container istio-proxy 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-38b47d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-38b47d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b Add eth0 [10.134.1.48/23] from ovn-kubernetes 21m Normal Pulled pod/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b Container created 21m Normal Started pod/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b Container started 21m Normal AddedInterface pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Add eth0 [10.133.0.210/23] from ovn-kubernetes 21m Normal Started pod/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m Container started 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m Add eth0 [10.134.1.47/23] from ovn-kubernetes 21m Normal Pulled pod/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m Container created 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj Stopping container payload-processing 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd Error creating: pods "payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-5f99e7: serviceaccount "payload-processing-e2e-ipp-a-5f99e7" not found 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7 Error creating: pods "payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-5f99e7: serviceaccount "payload-processing-e2e-ipp-a-5f99e7" not found 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd Created pod: payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-gtknt 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-5f99e7 Scaled up replica set payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd from 0 to 1 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-5f99e7 Scaled up replica set payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7 from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7 Created pod: payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-szq9v 21m Normal Pulled pod/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-gtknt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-szq9v Add eth0 [10.132.0.205/23] from ovn-kubernetes 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-szq9v Container started 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-szq9v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-gtknt Add eth0 [10.133.0.212/23] from ovn-kubernetes 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-szq9v Container created 21m Normal Created pod/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-gtknt Container created 21m Normal Started pod/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-gtknt Container started 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c Created pod: e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d 21m Normal ScalingReplicaSet deployment/e2e-ipp-b-fc3cb2-openshift-default Scaled up replica set e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c from 0 to 1 21m Normal AddedLabel gateway/e2e-ipp-b-fc3cb2 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-fc3cb2 21m Normal AddedLabel gateway/e2e-ipp-b-6cba49 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-6cba49 21m Normal ScalingReplicaSet deployment/e2e-ipp-b-6cba49-openshift-default Scaled up replica set e2e-ipp-b-6cba49-openshift-default-6c65f96df5 from 0 to 1 21m Normal AddedInterface pod/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d Add eth0 [10.132.0.206/23] from ovn-kubernetes 21m Normal AddedInterface pod/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp Add eth0 [10.134.1.49/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-6cba49-openshift-default-6c65f96df5 Created pod: e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd Error creating: pods "payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6cba49: serviceaccount "payload-processing-e2e-ipp-b-6cba49" not found 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd Created pod: payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-wskgt 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-6cba49 Scaled up replica set payload-pre-processing-e2e-ipp-b-6cba49-76d444555 from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-6cba49-76d444555 Created pod: payload-pre-processing-e2e-ipp-b-6cba49-76d444555-7qh5w 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-6cba49-76d444555 Error creating: pods "payload-pre-processing-e2e-ipp-b-6cba49-76d444555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6cba49: serviceaccount "payload-processing-e2e-ipp-b-6cba49" not found 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-6cba49 Scaled up replica set payload-processing-e2e-ipp-b-6cba49-65ffff6bdd from 0 to 1 20m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7bdc57c4cf from 4 to 3 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-6cba49-76d444555-7qh5w Add eth0 [10.133.0.214/23] from ovn-kubernetes 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-6cba49-76d444555-7qh5w Container started 20m Normal Killing pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Stopping container istio-proxy 20m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-xsffc 20m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-6cba49-76d444555-7qh5w Container created 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-6cba49-76d444555-7qh5w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55 Created pod: payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-fgsx4 20m Normal Started pod/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-wskgt Container started 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b Error creating: pods "payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-fc3cb2: serviceaccount "payload-processing-e2e-ipp-b-fc3cb2" not found 20m Normal Created pod/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-wskgt Container created 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55 Error creating: pods "payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-fc3cb2: serviceaccount "payload-processing-e2e-ipp-b-fc3cb2" not found 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b Created pod: payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-zwdzp 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls Stopping container payload-pre-processing 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-fc3cb2 Scaled up replica set payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55 from 0 to 1 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-wskgt Add eth0 [10.133.0.215/23] from ovn-kubernetes 20m Normal Pulled pod/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-wskgt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-fc3cb2 Scaled up replica set payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b from 0 to 1 20m Normal Pulled pod/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-zwdzp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Killing pod/payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c Stopping container payload-processing 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-fgsx4 Container started 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-fgsx4 Container created 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-fgsx4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-fgsx4 Add eth0 [10.133.0.216/23] from ovn-kubernetes 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-zwdzp Add eth0 [10.133.0.217/23] from ovn-kubernetes 20m Normal Started pod/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-zwdzp Container started 20m Normal Created pod/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-zwdzp Container created 20m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Deleted pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae06fe-openshift-default Scaled down replica set e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 from 10 to 8 20m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Deleted pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn 20m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-x2mfj Stopping container istio-proxy 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default New size: 8; reason: All metrics below target 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-5f99e7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-lsmcn Stopping container istio-proxy 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-xsffc Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-5f99e7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-f69104-698c567f7f-dmphf Readiness probe failed: timeout: failed to connect service "10.133.0.157:50052" within 1s: context deadline exceeded 20m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585 Deleted pod: e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-62f5d1-openshift-default Scaled down replica set e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585 from 3 to 2 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-62f5d1-openshift-default New size: 2; reason: All metrics below target 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-969b92-openshift-default New size: 3; reason: All metrics below target 20m Normal SuccessfulDelete replicaset/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f Deleted pod: e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt 20m Normal Killing pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ggqkt Stopping container istio-proxy 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-f69104-7cf4dc59cb-cb6sj Readiness probe failed: timeout: failed to connect service "10.133.0.158:50052" within 1s: context deadline exceeded 20m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-969b92-openshift-default Scaled down replica set e2e-shared-a-w2-969b92-openshift-default-b496c6b6f from 4 to 3 20m Normal Killing pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-qxdsm Stopping container istio-proxy 20m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-b-38b47d-65b5c477c9-25vls Readiness probe failed: timeout: failed to connect service "10.132.0.203:50052" within 1s: context deadline exceeded 20m Warning Unhealthy pod/payload-processing-e2e-ipp-b-38b47d-5d787c7c54-v5z7c Readiness probe failed: timeout: failed to connect service "10.133.0.204:50052" within 1s: context deadline exceeded 20m Normal Killing pod/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-a-f91e87-7bf7b54d6-7nvt8 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f91e87-7c488fc6f9-bkf7w Stopping container payload-pre-processing 20m Normal Killing pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-wx5ss Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-gfqf7 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w3-f69104-openshift-default-5c7f96bd7-cdglz Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-std5z Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-f91e87-openshift-default-6d7b498dff-zgj9m Stopping container istio-proxy 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-6c4f6f-69b667986-p2bwx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-6c4f6f-755bc8fb4c-jhs9s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-38b47d-65c9d667c6 Error creating: pods "payload-processing-e2e-ipp-b-38b47d-65c9d667c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-38b47d: serviceaccount "payload-processing-e2e-ipp-b-38b47d" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-38b47d-db764875 Created pod: payload-pre-processing-e2e-ipp-b-38b47d-db764875-k8wxg 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-38b47d Scaled up replica set payload-pre-processing-e2e-ipp-b-38b47d-db764875 from 0 to 1 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-38b47d Scaled up replica set payload-processing-e2e-ipp-b-38b47d-65c9d667c6 from 0 to 1 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-38b47d-db764875 Error creating: pods "payload-pre-processing-e2e-ipp-b-38b47d-db764875-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-38b47d: serviceaccount "payload-processing-e2e-ipp-b-38b47d" not found 20m Normal Killing pod/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb Stopping container payload-pre-processing 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-38b47d-65c9d667c6 Created pod: payload-processing-e2e-ipp-b-38b47d-65c9d667c6-snwt5 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-38b47d-65c9d667c6-snwt5 Add eth0 [10.133.0.223/23] from ovn-kubernetes 20m Normal Created pod/payload-processing-e2e-ipp-b-38b47d-65c9d667c6-snwt5 Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-b-38b47d-65c9d667c6-snwt5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-38b47d-db764875-k8wxg Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-38b47d-db764875-k8wxg Add eth0 [10.133.0.222/23] from ovn-kubernetes 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-38b47d-db764875-k8wxg Container created 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-38b47d-db764875-k8wxg Container started 20m Normal Started pod/payload-processing-e2e-ipp-b-38b47d-65c9d667c6-snwt5 Container started 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1a42e5-5f495b4bf9-bpmp7 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-1a42e5-8b4998c78-p9lfr Stopping container payload-processing 19m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh Stopping container istio-proxy 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Deleted pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-7sjxh 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Deleted pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n 19m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-npq7n Stopping container istio-proxy 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae06fe-openshift-default New size: 5; reason: All metrics below target 19m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae06fe-openshift-default Scaled down replica set e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 from 8 to 5 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787 Deleted pod: e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-hfl5f 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-b3593a-5465ffb8c9-7gpkb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-worker-w3-b3593a-58cf9fd4c8-t6j92 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-ipp-b-1a42e5-openshift-default-84779c8749-k8vc8 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-pgbtc Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Stopping container istio-proxy 19m Warning ProbeError pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-a-w4-6c4f6f-openshift-default-6bbb9d4f57-q44f2 Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl Stopping container payload-processing 19m Normal Killing pod/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-a-5f99e7-55fbd4b6cd-gtknt Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-5f99e7-74998646b7-szq9v Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 Stopping container istio-proxy 19m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-60e8f4-openshift-default Scaled down replica set e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c from 4 to 3 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-60e8f4-openshift-default New size: 3; reason: All metrics below target 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c Deleted pod: e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-4ngr2 19m Normal Created pod/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d Container created 19m Normal Pulled pod/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Pulled pod/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Started pod/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d Container started 19m Normal Killing pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-5w62n Stopping container istio-proxy 19m Normal Created pod/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp Container created 19m Normal Started pod/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp Container started 19m Normal Killing pod/e2e-shared-a-w0-bea3ef-openshift-default-784b4db987-p4np8 Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-bea3ef-7d644dff79-2mmvk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Started pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Container started 19m Normal Created pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Container created 19m Normal Pulled pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-bea3ef-5dc5b8c9b4-7tdcl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning ProbeError pod/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d Startup probe error: Get "http://10.132.0.206:15021/healthz/ready": dial tcp 10.132.0.206:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d Startup probe failed: Get "http://10.132.0.206:15021/healthz/ready": dial tcp 10.132.0.206:15021: connect: connection refused 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-fc3cb2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-fc3cb2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning Unhealthy pod/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp Startup probe failed: Get "http://10.134.1.49:15021/healthz/ready": dial tcp 10.134.1.49:15021: connect: connection refused 19m Warning ProbeError pod/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp Startup probe error: Get "http://10.134.1.49:15021/healthz/ready": dial tcp 10.134.1.49:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Startup probe failed: Get "http://10.133.0.210:15021/healthz/ready": dial tcp 10.133.0.210:15021: connect: connection refused 19m Warning ProbeError pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Startup probe error: Get "http://10.133.0.210:15021/healthz/ready": dial tcp 10.133.0.210:15021: connect: connection refused... 19m Normal Killing pod/payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj Stopping container payload-pre-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-fc3cb2-5978954f55-fgsx4 Stopping container payload-pre-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-6cba49-76d444555-7qh5w Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-fc3cb2-d6577bb7b-zwdzp Stopping container payload-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-6cba49-65ffff6bdd-wskgt Stopping container payload-processing 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-33e759-56db6984f9-tx88v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-33e759-86f4b57dc4-fksx4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-6cba49-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-6cba49-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Deleted pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Deleted pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-60e8f4-openshift-default New size: 2; reason: All metrics below target 19m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-60e8f4-openshift-default Scaled down replica set e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c from 3 to 2 19m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-16c1bb-openshift-default Scaled down replica set e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 from 10 to 6 19m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx Stopping container istio-proxy 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default New size: 6; reason: All metrics below target 19m Normal Killing pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Stopping container istio-proxy 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Deleted pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2zbhx 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c Deleted pod: e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-kvx4z 19m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Deleted pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q 18m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-8dszn Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w8t8t Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-ndf5q Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-62f5d1-5b9c4dc69f-hnzwj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-62f5d1-6649996454-bqvnx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-q524r Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w3-b3593a-openshift-default-c5596b489-kmmmn Stopping container istio-proxy 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-481249-6686b74599-lw59m Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-481249-6d59bcdc6b-f727l Stopping container payload-processing 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-5f99e7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-5f99e7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Killing pod/payload-processing-e2e-ipp-a-d1ae36-d65d966b6-zkv8m Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-d1ae36-676b8d9d9f-tttlf Stopping container payload-pre-processing 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-5f99e7 Scaled up replica set payload-processing-e2e-ipp-a-5f99e7-58f6878784 from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-5f99e7 Scaled up replica set payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f from 0 to 1 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f Error creating: pods "payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-5f99e7: serviceaccount "payload-processing-e2e-ipp-a-5f99e7" not found 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-5f99e7-58f6878784 Created pod: payload-processing-e2e-ipp-a-5f99e7-58f6878784-x95g8 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-5f99e7-58f6878784 Error creating: pods "payload-processing-e2e-ipp-a-5f99e7-58f6878784-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-5f99e7: serviceaccount "payload-processing-e2e-ipp-a-5f99e7" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f Created pod: payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-95pgv 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-6cba49-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-fc3cb2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Started pod/payload-processing-e2e-ipp-a-5f99e7-58f6878784-x95g8 Container started 18m Normal Created pod/payload-processing-e2e-ipp-a-5f99e7-58f6878784-x95g8 Container created 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-95pgv Add eth0 [10.132.0.212/23] from ovn-kubernetes 18m Normal AddedInterface pod/payload-processing-e2e-ipp-a-5f99e7-58f6878784-x95g8 Add eth0 [10.132.0.211/23] from ovn-kubernetes 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-95pgv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-6cba49-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-fc3cb2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Created pod/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-95pgv Container created 18m Normal Pulled pod/payload-processing-e2e-ipp-a-5f99e7-58f6878784-x95g8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-95pgv Container started 18m Normal Killing pod/e2e-ipp-a-481249-openshift-default-6c77499cf5-phgd9 Stopping container istio-proxy 18m Normal Killing pod/e2e-ipp-a-d1ae36-openshift-default-56d5b87b55-n9plm Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-57rgb Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2thxx Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-shared-b-w4-33e759-openshift-default-845bcbf7fc-2dghv Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf Created pod: payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-tqk48 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-6cba49 Scaled up replica set payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-6cba49 Scaled up replica set payload-processing-e2e-ipp-b-6cba49-9df45c854 from 0 to 1 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-6cba49-9df45c854 Error creating: pods "payload-processing-e2e-ipp-b-6cba49-9df45c854-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6cba49: serviceaccount "payload-processing-e2e-ipp-b-6cba49" not found 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf Error creating: pods "payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6cba49: serviceaccount "payload-processing-e2e-ipp-b-6cba49" not found 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-tqk48 Container started 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-tqk48 Add eth0 [10.132.0.213/23] from ovn-kubernetes 18m Normal Started pod/payload-processing-e2e-ipp-b-6cba49-9df45c854-hqw6c Container started 18m Normal Pulled pod/payload-processing-e2e-ipp-b-6cba49-9df45c854-hqw6c Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-6cba49-9df45c854 Created pod: payload-processing-e2e-ipp-b-6cba49-9df45c854-hqw6c 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-tqk48 Container created 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-tqk48 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-processing-e2e-ipp-b-6cba49-9df45c854-hqw6c Container created 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-6cba49-9df45c854-hqw6c Add eth0 [10.132.0.214/23] from ovn-kubernetes 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-fc3cb2 Scaled up replica set payload-processing-e2e-ipp-b-fc3cb2-58bc75d6 from 0 to 1 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6 Error creating: pods "payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-fc3cb2: serviceaccount "payload-processing-e2e-ipp-b-fc3cb2" not found 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-fc3cb2 Scaled up replica set payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f Created pod: payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-wz49m 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6 Created pod: payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-cppx7 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f Error creating: pods "payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-fc3cb2: serviceaccount "payload-processing-e2e-ipp-b-fc3cb2" not found 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-wz49m Add eth0 [10.132.0.215/23] from ovn-kubernetes 18m Normal Created pod/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-cppx7 Container created 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-wz49m Container started 18m Normal AddedLabel gateway/e2e-ipp-b-3e9921 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-3e9921 18m Normal Killing pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-nlc72 Stopping container istio-proxy 18m Normal Started pod/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-cppx7 Container started 18m Normal ScalingReplicaSet deployment/e2e-ipp-b-3e9921-openshift-default Scaled up replica set e2e-ipp-b-3e9921-openshift-default-7d98cc65c5 from 0 to 1 18m Normal SuccessfulCreate replicaset/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5 Created pod: e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 18m Normal Killing pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Stopping container istio-proxy 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-cppx7 Add eth0 [10.132.0.216/23] from ovn-kubernetes 18m Normal Pulled pod/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-cppx7 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-wz49m Container created 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-wz49m Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 Add eth0 [10.132.0.217/23] from ovn-kubernetes 18m Normal Killing pod/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd Stopping container payload-processing 18m Warning Unhealthy pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-shared-b-w0-62f5d1-openshift-default-69fc57b585-986c2 Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-3e9921 Scaled up replica set payload-pre-processing-e2e-ipp-b-3e9921-889685dbc from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc Created pod: payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-q7v6x 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc Error creating: pods "payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3e9921: serviceaccount "payload-processing-e2e-ipp-b-3e9921" not found 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-3e9921 Scaled up replica set payload-processing-e2e-ipp-b-3e9921-5f86b9565b from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-3e9921-5f86b9565b Created pod: payload-processing-e2e-ipp-b-3e9921-5f86b9565b-d5wpq 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-3e9921-5f86b9565b Error creating: pods "payload-processing-e2e-ipp-b-3e9921-5f86b9565b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3e9921: serviceaccount "payload-processing-e2e-ipp-b-3e9921" not found 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-q7v6x Add eth0 [10.132.0.218/23] from ovn-kubernetes 18m Normal Started pod/payload-processing-e2e-ipp-b-3e9921-5f86b9565b-d5wpq Container started 18m Normal Created pod/payload-processing-e2e-ipp-b-3e9921-5f86b9565b-d5wpq Container created 18m Normal Pulled pod/payload-processing-e2e-ipp-b-3e9921-5f86b9565b-d5wpq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-3e9921-5f86b9565b-d5wpq Add eth0 [10.134.1.55/23] from ovn-kubernetes 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-q7v6x Container started 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-q7v6x Container created 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-q7v6x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal ScalingReplicaSet deployment/e2e-ipp-a-5f99e7-openshift-default Scaled up replica set e2e-ipp-a-5f99e7-openshift-default-65d869d4b9 from 1 to 2 18m Normal SuccessfulCreate replicaset/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9 Created pod: e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-5f99e7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 18m Normal Started pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c Container started 18m Normal Pulled pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c Add eth0 [10.132.0.219/23] from ovn-kubernetes 18m Normal Created pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c Container created 18m Normal Killing pod/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt Stopping container payload-processing 18m Warning Unhealthy pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused 18m Warning ProbeError pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused... 18m Warning Unhealthy pod/payload-processing-e2e-worker-w4-6add35-7cd86547c6-gnkgd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-6add35-b8dcb9999-mhp86 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-pre-processing-e2e-ipp-b-38b47d-db764875-k8wxg Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-b-38b47d-65c9d667c6-snwt5 Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-b-6cba49-676cc5bbcf-tqk48 Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-b-6cba49-9df45c854-hqw6c Stopping container payload-processing 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-2b4105-755d7d6c47-tvwdr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-worker-w0-2b4105-85f86769d6-jqfkt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-ipp-b-6cba49-openshift-default-6c65f96df5-7nrwp Stopping container istio-proxy 17m Normal Killing pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-ipp-b-38b47d-openshift-default-649dd76fb5-8m5tz Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-3e9921-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-3e9921-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 Stopping container payload-pre-processing 17m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 Deleted pod: e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft 17m Normal Killing pod/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq Stopping container payload-processing 17m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-16c1bb-openshift-default Scaled down replica set e2e-shared-b-w1-16c1bb-openshift-default-7546866c77 from 6 to 5 17m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-h72ft Stopping container istio-proxy 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-16c1bb-openshift-default New size: 5; reason: All metrics below target 17m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-969b92-openshift-default Scaled down replica set e2e-shared-a-w2-969b92-openshift-default-b496c6b6f from 3 to 2 17m Normal SuccessfulDelete replicaset/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f Deleted pod: e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz 17m Normal Killing pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Stopping container istio-proxy 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-969b92-openshift-default New size: 2; reason: All metrics below target 17m Normal Killing pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-2fw7b Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-tb9fz Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Started pod/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 Container started 17m Normal Pulled pod/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Created pod/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 Container created 17m Warning Unhealthy pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal ScalingReplicaSet deployment/e2e-worker-w0-2b4105-openshift-default Scaled up replica set e2e-worker-w0-2b4105-openshift-default-d944d864f from 2 to 3 17m Normal AddedInterface pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w Add eth0 [10.132.0.221/23] from ovn-kubernetes 17m Warning ProbeError pod/e2e-shared-a-w2-969b92-openshift-default-b496c6b6f-ftc6x Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Pulled pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal SuccessfulCreate replicaset/e2e-worker-w0-2b4105-openshift-default-d944d864f Created pod: e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2b4105-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 17m Normal Started pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w Container started 17m Normal Created pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w Container created 17m Warning ProbeError pod/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 Startup probe error: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 Startup probe failed: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused 17m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w Startup probe failed: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused 17m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w Startup probe error: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused... 17m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-ckw5w Stopping container istio-proxy 17m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Stopping container istio-proxy 17m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 Stopping container payload-pre-processing 17m Normal Killing pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-9wxjh Stopping container istio-proxy 17m Normal Killing pod/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-3e9921-889685dbc-q7v6x Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-3e9921-5f86b9565b-d5wpq Stopping container payload-processing 17m Warning Unhealthy pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-worker-w0-2b4105-openshift-default-d944d864f-xlbkh Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-969b92-75f748cc6b-bq5bq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-969b92-57ff684f47-645k9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-ea2547-5fbb8b67b8-q6n4b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-ea2547-f846cf7c5-8mwn7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Stopping container istio-proxy 17m Normal Killing pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Stopping container istio-proxy 17m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-9l9cp Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-worker-w4-6add35-openshift-default-9c5d9d5ff-vwbjv Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal Killing pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-vwhcx Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-shared-a-w6-ea2547-openshift-default-d6bcbb6d7-xcq9h Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-3e9921-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-3e9921-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wllzj Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-wj4kp Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dx9j5 Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-dc6t4 Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Stopping container istio-proxy 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-ae06fe-96bfc5f55-4fjl4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-3e9921 Scaled up replica set payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87 from 0 to 1 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-ae06fe-77c7cf7cfb-s4qwn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-3e9921 Scaled up replica set payload-processing-e2e-ipp-b-3e9921-6555c58b98 from 0 to 1 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-w94vx Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87 Created pod: payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-w94vx 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87 Error creating: pods "payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3e9921: serviceaccount "payload-processing-e2e-ipp-b-3e9921" not found 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-w94vx Add eth0 [10.132.0.222/23] from ovn-kubernetes 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-3e9921-6555c58b98 Error creating: pods "payload-processing-e2e-ipp-b-3e9921-6555c58b98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3e9921: serviceaccount "payload-processing-e2e-ipp-b-3e9921" not found 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-3e9921-6555c58b98 Created pod: payload-processing-e2e-ipp-b-3e9921-6555c58b98-k6f8c 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-w94vx Container created 16m Normal Killing pod/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 Stopping container payload-pre-processing 16m Normal Pulled pod/payload-processing-e2e-ipp-b-3e9921-6555c58b98-k6f8c Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-w94vx Container started 16m Normal Created pod/payload-processing-e2e-ipp-b-3e9921-6555c58b98-k6f8c Container created 16m Normal Started pod/payload-processing-e2e-ipp-b-3e9921-6555c58b98-k6f8c Container started 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-3e9921-6555c58b98-k6f8c Add eth0 [10.134.1.65/23] from ovn-kubernetes 16m Warning Unhealthy pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-b-w2-ae06fe-openshift-default-56454b6787-8jpk4 Readiness probe error: HTTP probe failed with statuscode: 503... 16m Normal Killing pod/payload-pre-processing-e2e-ipp-a-abf8ae-7c4d75f895-9rs72 Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-a-abf8ae-7cccbf8ff6-b6k7b Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq Stopping container payload-processing 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-60e8f4-69fc4846cb-7whf8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-60e8f4-6b8d55bf6c-wcc95 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-dtkfc Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w6-60e8f4-openshift-default-5fffb9894c-djc7v Stopping container istio-proxy 16m Warning Unhealthy pod/payload-processing-e2e-worker-w2-40ac41-776bf85c68-r5qdq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-40ac41-86d7dfdfb7-4hn8c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-pre-processing-e2e-ipp-a-5f99e7-5454df895f-95pgv Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-a-5f99e7-58f6878784-x95g8 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq Stopping container payload-processing 16m Warning ProbeError pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Readiness probe error: HTTP probe failed with statuscode: 503... 16m Normal Killing pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Stopping container istio-proxy 16m Warning Unhealthy pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-qv88m Readiness probe failed: HTTP probe failed with statuscode: 503 16m Normal Killing pod/e2e-ipp-a-5f99e7-openshift-default-65d869d4b9-jc28c Stopping container istio-proxy 16m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-tqxtm Stopping container istio-proxy 16m Normal Killing pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Stopping container istio-proxy 16m Warning ProbeError pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/e2e-worker-w2-40ac41-openshift-default-5c6c78d655-765q4 Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning Unhealthy pod/payload-processing-e2e-worker-w6-5062f6-54896b9c4d-986mq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-5062f6-5c47cc4b8c-rgtvm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-3e9921-dd76f5d87-w94vx Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-b-3e9921-6555c58b98-k6f8c Stopping container payload-processing 16m Normal Killing pod/e2e-ipp-a-abf8ae-openshift-default-578749ddbf-nqnr8 Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-b-3e9921-openshift-default-7d98cc65c5-xr8j7 Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-tj8nz Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w6-5062f6-openshift-default-68b95c9cff-jj77c Stopping container istio-proxy 15m Normal Killing pod/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-b-fc3cb2-58bc75d6-cppx7 Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-fc3cb2-645847cb7f-wz49m Stopping container payload-pre-processing 15m Normal Killing pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-5sr8q Stopping container istio-proxy 15m Warning Unhealthy pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-shared-a-w1-95492e-openshift-default-7bd676f4b9-tb7qb Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/e2e-ipp-b-fc3cb2-openshift-default-69bbd5978c-jx55d Stopping container istio-proxy 15m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-95492e-6cbcb58bc4-qdj6v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-95492e-c44db6c87-vdr9w Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-d01638-74f77f4787-j758b Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-a-w5-d01638-5668596858-chl8d Stopping container payload-processing 15m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-dll5h Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-w2dwq Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-2qx9n Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-zjvdb Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w1-16c1bb-openshift-default-7546866c77-l9q62 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Stopping container istio-proxy 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-16c1bb-74c9779647-bvlps Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-16c1bb-59d75f5c74-v4bth Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning ProbeError pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/e2e-shared-a-w5-d01638-openshift-default-884b45857-7kkmx Readiness probe failed: HTTP probe failed with statuscode: 503 14m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 14m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-wvbtj 14m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 14m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-wvbtj Stopping container discovery 14m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff Stopping container payload-processing 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-8e24a5-5f6c6bcd5d-h5dff Readiness probe failed: timeout: failed to connect service "10.132.0.178:50052" within 1s: context deadline exceeded 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-8e24a5-56d4d96b5f-4lq42 Readiness probe failed: timeout: failed to connect service "10.134.1.13:50052" within 1s: context deadline exceeded 14m Normal Killing pod/e2e-shared-b-w5-8e24a5-openshift-default-549fd6595c-2stl2 Stopping container istio-proxy 14m Warning Unhealthy pod/payload-processing-e2e-worker-w1-beb8d5-cb6dcb8cb-cx87m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-beb8d5-9bf795695-kmhxr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h Stopping container payload-pre-processing 14m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-z4hdh Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w1-beb8d5-openshift-default-fc65648b7-tbhdm Stopping container istio-proxy 14m Warning Unhealthy pod/payload-processing-e2e-worker-w5-89a676-5679c8db-2tb86 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-89a676-6d9c994f55-5vk7h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-fcx7s Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w5-89a676-openshift-default-6bdc5cc785-kzw7l Stopping container istio-proxy 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 3 to 6 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 13m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Container created 13m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Container started 13m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Container started 13m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Container created 13m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Container started 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Add eth0 [10.134.1.79/23] from ovn-kubernetes 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Add eth0 [10.132.0.226/23] from ovn-kubernetes 13m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Container created 13m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Add eth0 [10.133.0.235/23] from ovn-kubernetes 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Startup probe error: Get "http://10.132.0.226:15021/healthz/ready": dial tcp 10.132.0.226:15021: connect: connection refused... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Startup probe error: Get "http://10.134.1.79:15021/healthz/ready": dial tcp 10.134.1.79:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Startup probe failed: Get "http://10.132.0.226:15021/healthz/ready": dial tcp 10.132.0.226:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Startup probe error: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Startup probe failed: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Startup probe failed: Get "http://10.134.1.79:15021/healthz/ready": dial tcp 10.134.1.79:15021: connect: connection refused 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Startup probe failed: Get "http://10.132.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Startup probe error: Get "http://10.133.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Startup probe failed: Get "http://10.134.1.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-nzbt2 Startup probe failed: Get "http://10.133.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Startup probe error: Get "http://10.132.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-zpcs2 Startup probe error: Get "http://10.134.1.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 6 to 7 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-885k9 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Add eth0 [10.132.0.227/23] from ovn-kubernetes 13m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Container created 13m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Container started 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Startup probe error: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Startup probe failed: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused 12m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-4fglz 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7bdc57c4cf Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-pc67m 12m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 7 to 10 12m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Add eth0 [10.133.0.236/23] from ovn-kubernetes 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Add eth0 [10.132.0.229/23] from ovn-kubernetes 12m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Container started 12m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Container started 12m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Container created 12m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Container created 12m Normal Created pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Container created 12m Normal Pulled pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Add eth0 [10.134.1.83/23] from ovn-kubernetes 12m Normal Started pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Container started 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Startup probe error: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Startup probe error: Get "http://10.134.1.83:15021/healthz/ready": dial tcp 10.134.1.83:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Startup probe failed: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Startup probe failed: Get "http://10.134.1.83:15021/healthz/ready": dial tcp 10.134.1.83:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Startup probe failed: Get "http://10.133.0.236:15021/healthz/ready": dial tcp 10.133.0.236:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Startup probe error: Get "http://10.133.0.236:15021/healthz/ready": dial tcp 10.133.0.236:15021: connect: connection refused... 11m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-qbkz5 Stopping container discovery 11m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-qbkz5 11m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 11m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Readiness probe failed: Get "http://10.134.1.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Readiness probe error: Get "http://10.132.0.229:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Readiness probe error: Get "http://10.133.0.236:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-hjz7l Readiness probe failed: Get "http://10.133.0.236:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Readiness probe error: Get "http://10.132.0.227:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-4fglz Readiness probe failed: Get "http://10.132.0.229:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-885k9 Readiness probe failed: Get "http://10.132.0.227:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-pc67m Readiness probe error: Get "http://10.134.1.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Readiness probe failed: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 10m Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-7nffr Readiness probe error: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 9m3s Normal SuccessfulCreate replicaset/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997 Created pod: e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz 9m3s Normal ScalingReplicaSet deployment/e2e-worker-single-main-ff1bf5-openshift-default Scaled up replica set e2e-worker-single-main-ff1bf5-openshift-default-b756b9997 from 0 to 1 9m3s Normal AddedLabel gateway/e2e-worker-single-main-ff1bf5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-ff1bf5 9m2s Normal AddedInterface pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Add eth0 [10.132.0.230/23] from ovn-kubernetes 8m57s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-ff1bf5 Scaled up replica set payload-pre-processing-e2e-worker-single-main-ff1bf5-5b458896bc from 0 to 1 8m57s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b458896bc Created pod: payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb 8m57s Normal Pulled pod/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m57s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt Add eth0 [10.132.0.231/23] from ovn-kubernetes 8m57s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97b Created pod: payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt 8m57s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97b Error creating: pods "payload-processing-e2e-worker-single-main-ff1bf5-664d69d97b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ff1bf5: serviceaccount "payload-processing-e2e-worker-single-main-ff1bf5" not found 8m57s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b458896bc Error creating: pods "payload-pre-processing-e2e-worker-single-main-ff1bf5-5b458896bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ff1bf5: serviceaccount "payload-processing-e2e-worker-single-main-ff1bf5" not found 8m57s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-ff1bf5 Scaled up replica set payload-processing-e2e-worker-single-main-ff1bf5-664d69d97b from 0 to 1 8m56s Normal Started pod/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt Container started 8m56s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb Add eth0 [10.132.0.232/23] from ovn-kubernetes 8m56s Normal Created pod/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt Container created 8m56s Normal Started pod/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb Container started 8m56s Normal Created pod/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb Container created 8m56s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m48s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-ff1bf5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m48s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-ff1bf5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m44s Warning Unhealthy pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Readiness probe failed: Get "http://10.132.0.230:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m44s Warning ProbeError pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Readiness probe error: Get "http://10.132.0.230:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m43s Normal Started pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Container started 8m43s Normal Pulled pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m43s Normal Created pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Container created 8m38s Warning ProbeError pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Startup probe error: Get "http://10.132.0.230:15021/healthz/ready": dial tcp 10.132.0.230:15021: connect: connection refused... 8m38s Warning Unhealthy pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Startup probe failed: Get "http://10.132.0.230:15021/healthz/ready": dial tcp 10.132.0.230:15021: connect: connection refused 8m18s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-ff1bf5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m18s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-ff1bf5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m57s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb Stopping container payload-pre-processing 7m57s Normal Killing pod/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt Stopping container payload-processing 7m47s Normal Killing pod/e2e-worker-single-main-ff1bf5-openshift-default-b756b9997-vwhkz Stopping container istio-proxy 7m35s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-ff1bf5-664d69d97fj7nt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m35s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-ff1bf5-5b45875mvb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m29s Normal ScalingReplicaSet deployment/e2e-worker-single-main-7e040d-openshift-default Scaled up replica set e2e-worker-single-main-7e040d-openshift-default-85b9d9878f from 0 to 1 5m29s Normal SuccessfulCreate replicaset/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f Created pod: e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq 5m29s Normal AddedLabel gateway/e2e-worker-single-main-7e040d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-7e040d 5m28s Normal AddedInterface pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Add eth0 [10.134.1.99/23] from ovn-kubernetes 5m23s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4f9cc55 Created pod: payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 5m23s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-7e040d-6d656b46d Error creating: pods "payload-processing-e2e-worker-single-main-7e040d-6d656b46d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-7e040d: serviceaccount "payload-processing-e2e-worker-single-main-7e040d" not found 5m23s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 Add eth0 [10.134.1.101/23] from ovn-kubernetes 5m23s Normal Pulled pod/payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m23s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 Add eth0 [10.133.0.240/23] from ovn-kubernetes 5m23s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-7e040d Scaled up replica set payload-pre-processing-e2e-worker-single-main-7e040d-5cb4f9cc55 from 0 to 1 5m23s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4f9cc55 Error creating: pods "payload-pre-processing-e2e-worker-single-main-7e040d-5cb4f9cc55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-7e040d: serviceaccount "payload-processing-e2e-worker-single-main-7e040d" not found 5m23s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-7e040d-6d656b46d Created pod: payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 5m23s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-7e040d Scaled up replica set payload-processing-e2e-worker-single-main-7e040d-6d656b46d from 0 to 1 5m22s Normal Started pod/payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 Container started 5m22s Normal Started pod/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 Container started 5m22s Normal Created pod/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 Container created 5m22s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m22s Normal Created pod/payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 Container created 5m11s Warning Unhealthy pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Readiness probe failed: Get "http://10.134.1.99:15021/healthz/ready": read tcp 10.134.0.2:39396->10.134.1.99:15021: read: connection reset by peer 5m11s Warning ProbeError pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Readiness probe error: Get "http://10.134.1.99:15021/healthz/ready": read tcp 10.134.0.2:39396->10.134.1.99:15021: read: connection reset by peer... 5m10s Normal Pulled pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m10s Normal Created pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Container created 5m10s Normal Started pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Container started 5m5s Warning Unhealthy pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Startup probe failed: Get "http://10.134.1.99:15021/healthz/ready": dial tcp 10.134.1.99:15021: connect: connection refused 5m5s Warning ProbeError pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Startup probe error: Get "http://10.134.1.99:15021/healthz/ready": dial tcp 10.134.1.99:15021: connect: connection refused... 4m44s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-7e040d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m44s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-7e040d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m27s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 Stopping container payload-pre-processing 4m27s Normal Killing pod/payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 Stopping container payload-processing 4m19s Normal Killing pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Stopping container istio-proxy 4m19s Warning ProbeError pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Readiness probe error: HTTP probe failed with statuscode: 503... 4m19s Warning Unhealthy pod/e2e-worker-single-main-7e040d-openshift-default-85b9d9878f65zcq Readiness probe failed: HTTP probe failed with statuscode: 503 4m11s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-7e040d-6d656b46dsp927 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m11s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-7e040d-5cb4fzrxt6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m9s Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Readiness probe failed: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 4m9s Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-5bzrb Readiness probe error: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 106s Warning ProbeError pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Readiness probe error: Get "http://10.132.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 106s Warning Unhealthy pod/maas-default-gateway-openshift-default-7bdc57c4cf-sq9nj Readiness probe failed: Get "http://10.132.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)