=== events in openshift-ingress (2026-10-30T15:11:14Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 26m Normal Scheduled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv Successfully assigned openshift-ingress/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv to ip-10-0-133-118.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n to ip-10-0-131-200.ec2.internal 57m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 to ip-10-0-133-118.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-b-1d3696-b6fbdfb88-wdqxq Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-1d3696-b6fbdfb88-wdqxq to ip-10-0-131-200.ec2.internal 48m Normal Scheduled pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Successfully assigned openshift-ingress/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v to ip-10-0-140-105.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 Successfully assigned openshift-ingress/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 to ip-10-0-140-105.ec2.internal 65m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s to ip-10-0-131-200.ec2.internal 65m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b to ip-10-0-133-118.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 to ip-10-0-131-200.ec2.internal 58m Normal Scheduled pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Successfully assigned openshift-ingress/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn to ip-10-0-140-105.ec2.internal 59m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 to ip-10-0-133-118.ec2.internal 32m Normal Scheduled pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Successfully assigned openshift-ingress/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m to ip-10-0-131-200.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 to ip-10-0-131-200.ec2.internal 86m Warning FailedScheduling pod/router-default-659879fb99-pjvcq no nodes available to schedule pods 83m Warning FailedScheduling pod/router-default-659879fb99-pjvcq no nodes available to schedule pods 82m Normal Scheduled pod/router-default-659879fb99-pjvcq Successfully assigned openshift-ingress/router-default-659879fb99-pjvcq to ip-10-0-140-105.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv to ip-10-0-131-200.ec2.internal 30m Normal Scheduled pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Successfully assigned openshift-ingress/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 to ip-10-0-133-118.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-97sdt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-97sdt to ip-10-0-140-105.ec2.internal 66m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t to ip-10-0-131-200.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj to ip-10-0-133-118.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-kcz62 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-kcz62 to ip-10-0-140-105.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-h2hqx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-h2hqx to ip-10-0-133-118.ec2.internal 64m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 to ip-10-0-140-105.ec2.internal 57m Normal Scheduled pod/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 Successfully assigned openshift-ingress/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j to ip-10-0-131-200.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-qkgkv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-qkgkv to ip-10-0-131-200.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-prc47 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-prc47 to ip-10-0-133-118.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-dr679 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-dr679 to ip-10-0-131-200.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 to ip-10-0-140-105.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-lqzdk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-lqzdk to ip-10-0-140-105.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq to ip-10-0-140-105.ec2.internal 64m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg to ip-10-0-133-118.ec2.internal 61m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb to ip-10-0-131-200.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h to ip-10-0-131-200.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq to ip-10-0-140-105.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl to ip-10-0-131-200.ec2.internal 53m Normal Scheduled pod/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v Successfully assigned openshift-ingress/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v to ip-10-0-133-118.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 to ip-10-0-140-105.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf to ip-10-0-131-200.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-xlcmh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-xlcmh to ip-10-0-140-105.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-jdqzn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-jdqzn to ip-10-0-133-118.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k to ip-10-0-140-105.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-gw4r5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-gw4r5 to ip-10-0-131-200.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-3b95dd-844587bbf8-6rq9s Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-3b95dd-844587bbf8-6rq9s to ip-10-0-133-118.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q to ip-10-0-140-105.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr to ip-10-0-131-200.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc to ip-10-0-133-118.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh to ip-10-0-140-105.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh to ip-10-0-131-200.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv to ip-10-0-133-118.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-cwgl2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-cwgl2 to ip-10-0-140-105.ec2.internal 51m Normal Scheduled pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Successfully assigned openshift-ingress/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 to ip-10-0-133-118.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-z49d4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-z49d4 to ip-10-0-140-105.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh to ip-10-0-140-105.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp to ip-10-0-140-105.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw to ip-10-0-140-105.ec2.internal 32m Normal Scheduled pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc Successfully assigned openshift-ingress/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc to ip-10-0-131-200.ec2.internal 65m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 to ip-10-0-133-118.ec2.internal 64m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 to ip-10-0-140-105.ec2.internal 61m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq to ip-10-0-133-118.ec2.internal 61m Normal Scheduled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Successfully assigned openshift-ingress/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf to ip-10-0-140-105.ec2.internal 60m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld to ip-10-0-140-105.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 to ip-10-0-133-118.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w to ip-10-0-140-105.ec2.internal 66m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn to ip-10-0-131-200.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-5cv2v Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-5cv2v to ip-10-0-133-118.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-api-b-aab379-54cb856f5-6bvr9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-aab379-54cb856f5-6bvr9 to ip-10-0-133-118.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-m54k4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-m54k4 to ip-10-0-131-200.ec2.internal 54m Normal Scheduled pod/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p Successfully assigned openshift-ingress/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p to ip-10-0-133-118.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-pkl7b Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-pkl7b to ip-10-0-131-200.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-a-d74808-645df45b6c-h4f6c Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d74808-645df45b6c-h4f6c to ip-10-0-131-200.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-a-d727dc-db644f5ff-cg8nn Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d727dc-db644f5ff-cg8nn to ip-10-0-131-200.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-a-972fee-fdbb7989f-mdfj4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-972fee-fdbb7989f-mdfj4 to ip-10-0-140-105.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-4mkxk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-4mkxk to ip-10-0-140-105.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-5s7b7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-5s7b7 to ip-10-0-140-105.ec2.internal 65m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp to ip-10-0-131-200.ec2.internal 31m Normal Scheduled pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Successfully assigned openshift-ingress/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j to ip-10-0-133-118.ec2.internal 40m Normal Scheduled pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Successfully assigned openshift-ingress/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq to ip-10-0-131-200.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Successfully assigned openshift-ingress/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh to ip-10-0-133-118.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj to ip-10-0-131-200.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp to ip-10-0-133-118.ec2.internal 54m Normal Scheduled pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Successfully assigned openshift-ingress/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-00926c-77f9968595-6zj5z Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-00926c-77f9968595-6zj5z to ip-10-0-133-118.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-9w8xf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-9w8xf to ip-10-0-131-200.ec2.internal 45m Normal Scheduled pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl Successfully assigned openshift-ingress/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl to ip-10-0-133-118.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt to ip-10-0-131-200.ec2.internal 49m Normal Scheduled pod/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv Successfully assigned openshift-ingress/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv to ip-10-0-131-200.ec2.internal 13m Normal Scheduled pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Successfully assigned openshift-ingress/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx to ip-10-0-131-200.ec2.internal 31m Normal Scheduled pod/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 Successfully assigned openshift-ingress/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 to ip-10-0-140-105.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 to ip-10-0-140-105.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-ait-25917498-769cb747f9-hhxfb Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-25917498-769cb747f9-hhxfb to ip-10-0-131-200.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs to ip-10-0-131-200.ec2.internal 35m Normal Scheduled pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl Successfully assigned openshift-ingress/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl to ip-10-0-133-118.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-b-93549e-56ff978bf6-qh2c5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-93549e-56ff978bf6-qh2c5 to ip-10-0-140-105.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z to ip-10-0-133-118.ec2.internal 45m Normal Scheduled pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Successfully assigned openshift-ingress/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk to ip-10-0-133-118.ec2.internal 53m Normal Scheduled pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Successfully assigned openshift-ingress/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 to ip-10-0-133-118.ec2.internal 35m Normal Scheduled pod/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx Successfully assigned openshift-ingress/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx to ip-10-0-133-118.ec2.internal 68m Normal Scheduled pod/payload-pre-processing-7b98bb678c-g99dv Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-g99dv to ip-10-0-133-118.ec2.internal 40m Normal Scheduled pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Successfully assigned openshift-ingress/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj to ip-10-0-140-105.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n to ip-10-0-133-118.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-ct6xg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-ct6xg to ip-10-0-131-200.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-7z9qv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-7z9qv to ip-10-0-140-105.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-gfd5c Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-gfd5c to ip-10-0-140-105.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-4wb24 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-4wb24 to ip-10-0-140-105.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-dmk7s Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-dmk7s to ip-10-0-140-105.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 to ip-10-0-131-200.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn to ip-10-0-131-200.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-hmlcp Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-hmlcp to ip-10-0-140-105.ec2.internal 13m Normal Scheduled pod/payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n to ip-10-0-131-200.ec2.internal 9m45s Normal Scheduled pod/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm to ip-10-0-131-200.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 to ip-10-0-131-200.ec2.internal 51m Normal Scheduled pod/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt Successfully assigned openshift-ingress/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt to ip-10-0-133-118.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h to ip-10-0-133-118.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-4j9qs Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-4j9qs to ip-10-0-133-118.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-wpppq Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-wpppq to ip-10-0-133-118.ec2.internal 43m Normal Scheduled pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Successfully assigned openshift-ingress/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj to ip-10-0-131-200.ec2.internal 59m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 to ip-10-0-133-118.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw to ip-10-0-140-105.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q to ip-10-0-140-105.ec2.internal 65m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 to ip-10-0-140-105.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx Successfully assigned openshift-ingress/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx to ip-10-0-140-105.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 to ip-10-0-131-200.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 to ip-10-0-131-200.ec2.internal 33m Normal Scheduled pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z Successfully assigned openshift-ingress/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z to ip-10-0-133-118.ec2.internal 49m Normal Scheduled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Successfully assigned openshift-ingress/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd to ip-10-0-131-200.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn to ip-10-0-131-200.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 to ip-10-0-131-200.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw to ip-10-0-131-200.ec2.internal 47m Normal Scheduled pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Successfully assigned openshift-ingress/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 to ip-10-0-133-118.ec2.internal 30m Normal Scheduled pod/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr Successfully assigned openshift-ingress/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr to ip-10-0-133-118.ec2.internal 10m Normal Scheduled pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Successfully assigned openshift-ingress/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 to ip-10-0-131-200.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 to ip-10-0-131-200.ec2.internal 66m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj to ip-10-0-133-118.ec2.internal 50m Normal Scheduled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Successfully assigned openshift-ingress/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z to ip-10-0-131-200.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z to ip-10-0-131-200.ec2.internal 33m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 to ip-10-0-131-200.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-kb9kl Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-kb9kl to ip-10-0-131-200.ec2.internal 32m Normal Scheduled pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Successfully assigned openshift-ingress/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js to ip-10-0-140-105.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 to ip-10-0-140-105.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr to ip-10-0-131-200.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Successfully assigned openshift-ingress/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm to ip-10-0-131-200.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd to ip-10-0-131-200.ec2.internal 50m Normal Scheduled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Successfully assigned openshift-ingress/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 to ip-10-0-140-105.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn to ip-10-0-140-105.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Successfully assigned openshift-ingress/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk to ip-10-0-140-105.ec2.internal 64m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls to ip-10-0-140-105.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Successfully assigned openshift-ingress/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 to ip-10-0-133-118.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w to ip-10-0-140-105.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc to ip-10-0-131-200.ec2.internal 65m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-ww87s Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-ww87s to ip-10-0-133-118.ec2.internal 49m Normal Scheduled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Successfully assigned openshift-ingress/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc to ip-10-0-133-118.ec2.internal 68m Normal Scheduled pod/payload-processing-5f78687d5b-9dw4d Successfully assigned openshift-ingress/payload-processing-5f78687d5b-9dw4d to ip-10-0-133-118.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 Successfully assigned openshift-ingress/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 to ip-10-0-140-105.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Successfully assigned openshift-ingress/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 to ip-10-0-133-118.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm to ip-10-0-140-105.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m to ip-10-0-140-105.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-qrkwf Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-qrkwf to ip-10-0-133-118.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs to ip-10-0-133-118.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t Successfully assigned openshift-ingress/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t to ip-10-0-133-118.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-x5f5d Successfully assigned openshift-ingress/payload-processing-e2e-mt-9dba4af9-6d98fd4964-x5f5d to ip-10-0-133-118.ec2.internal 50m Normal Scheduled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Successfully assigned openshift-ingress/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 to ip-10-0-140-105.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-wstfm to ip-10-0-133-118.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr Successfully assigned openshift-ingress/payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr to ip-10-0-140-105.ec2.internal 32m Normal Scheduled pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs Successfully assigned openshift-ingress/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 to ip-10-0-133-118.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js Successfully assigned openshift-ingress/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js to ip-10-0-133-118.ec2.internal 26m Normal Scheduled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f Successfully assigned openshift-ingress/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f to ip-10-0-131-200.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-mt-01955278-7d4f98b5fd-ltzh8 Successfully assigned openshift-ingress/payload-processing-e2e-mt-01955278-7d4f98b5fd-ltzh8 to ip-10-0-131-200.ec2.internal 26m Normal Scheduled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j Successfully assigned openshift-ingress/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j to ip-10-0-131-200.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-kgnb7 Successfully assigned openshift-ingress/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-kgnb7 to ip-10-0-133-118.ec2.internal 52m Normal Scheduled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Successfully assigned openshift-ingress/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr to ip-10-0-133-118.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-ait-25917498-5b656767dd-2xdq5 Successfully assigned openshift-ingress/payload-processing-e2e-ait-25917498-5b656767dd-2xdq5 to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l to ip-10-0-133-118.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q Successfully assigned openshift-ingress/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q to ip-10-0-131-200.ec2.internal 39m Normal Scheduled pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z Successfully assigned openshift-ingress/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z to ip-10-0-140-105.ec2.internal 66m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md to ip-10-0-133-118.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-d05e21-85db5c75d6-kg8b9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-d05e21-85db5c75d6-kg8b9 to ip-10-0-140-105.ec2.internal 64m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk to ip-10-0-131-200.ec2.internal 56m Normal Scheduled pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Successfully assigned openshift-ingress/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b to ip-10-0-140-105.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc to ip-10-0-140-105.ec2.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx to ip-10-0-131-200.ec2.internal 41m Normal Scheduled pod/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 Successfully assigned openshift-ingress/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 to ip-10-0-140-105.ec2.internal 60m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff to ip-10-0-131-200.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-ait-4522b19c-b46b9b49d-4ddlh Successfully assigned openshift-ingress/payload-processing-e2e-ait-4522b19c-b46b9b49d-4ddlh to ip-10-0-131-200.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-xzzzk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-xzzzk to ip-10-0-131-200.ec2.internal 66m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb to ip-10-0-133-118.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-a-2b206b-55f8fbf4-lpgqq Successfully assigned openshift-ingress/payload-processing-e2e-api-a-2b206b-55f8fbf4-lpgqq to ip-10-0-131-200.ec2.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-swl4n to ip-10-0-133-118.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-mt-01955278-f55689567-97lbq Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-01955278-f55689567-97lbq to ip-10-0-131-200.ec2.internal 18m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn to ip-10-0-133-118.ec2.internal 26m Normal Scheduled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt Successfully assigned openshift-ingress/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt to ip-10-0-140-105.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-bcc710-75c6df84d-h8h7n Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-bcc710-75c6df84d-h8h7n to ip-10-0-133-118.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-bcc710-7478845cd-x7lwp Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-bcc710-7478845cd-x7lwp to ip-10-0-140-105.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-90b406-5f96f96db6-dwhwj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-90b406-5f96f96db6-dwhwj to ip-10-0-140-105.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w Successfully assigned openshift-ingress/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w to ip-10-0-133-118.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-90b406-548d97d98d-tjsf4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-90b406-548d97d98d-tjsf4 to ip-10-0-140-105.ec2.internal 52m Normal Scheduled pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Successfully assigned openshift-ingress/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld to ip-10-0-133-118.ec2.internal 67m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s to ip-10-0-140-105.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl to ip-10-0-131-200.ec2.internal 18m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 to ip-10-0-140-105.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-n8p5d Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-n8p5d to ip-10-0-140-105.ec2.internal 33m Normal Scheduled pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp Successfully assigned openshift-ingress/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp to ip-10-0-131-200.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-api-a-4bf33a-6bb59459b-4qdp6 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-4bf33a-6bb59459b-4qdp6 to ip-10-0-140-105.ec2.internal 63m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx to ip-10-0-140-105.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx Successfully assigned openshift-ingress/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx to ip-10-0-140-105.ec2.internal 60m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp to ip-10-0-140-105.ec2.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq to ip-10-0-131-200.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-b-460b0b-86df895954-nvbqg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-460b0b-86df895954-nvbqg to ip-10-0-131-200.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-b-460b0b-55d85f88fb-7rkhm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-460b0b-55d85f88fb-7rkhm to ip-10-0-140-105.ec2.internal 65m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb to ip-10-0-140-105.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-vp88x Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-vp88x to ip-10-0-140-105.ec2.internal 9m45s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j to ip-10-0-131-200.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-9mjfq Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-9mjfq to ip-10-0-140-105.ec2.internal 52m Normal Scheduled pod/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc Successfully assigned openshift-ingress/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc to ip-10-0-133-118.ec2.internal 67m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-hszvg to ip-10-0-131-200.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-h5tsw Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-h5tsw to ip-10-0-133-118.ec2.internal 19m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b to ip-10-0-131-200.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-dsk2f Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-dsk2f to ip-10-0-133-118.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr to ip-10-0-140-105.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 to ip-10-0-140-105.ec2.internal 18m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-fb779 to ip-10-0-133-118.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp to ip-10-0-131-200.ec2.internal 18m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 to ip-10-0-140-105.ec2.internal 20m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 to ip-10-0-133-118.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh to ip-10-0-131-200.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk to ip-10-0-131-200.ec2.internal 18m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x to ip-10-0-131-200.ec2.internal 55m Normal Scheduled pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Successfully assigned openshift-ingress/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h to ip-10-0-133-118.ec2.internal 48m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt to ip-10-0-131-200.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-a-972fee-546d4b6686-svw78 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-972fee-546d4b6686-svw78 to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr to ip-10-0-133-118.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf to ip-10-0-133-118.ec2.internal 54m Normal Scheduled pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b Successfully assigned openshift-ingress/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b to ip-10-0-140-105.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 to ip-10-0-131-200.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-a-d727dc-bc4fc7c88-6sw2f Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d727dc-bc4fc7c88-6sw2f to ip-10-0-131-200.ec2.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 to ip-10-0-133-118.ec2.internal 47m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds to ip-10-0-131-200.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc to ip-10-0-140-105.ec2.internal 20m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx to ip-10-0-131-200.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s Successfully assigned openshift-ingress/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s to ip-10-0-133-118.ec2.internal 60m Normal Scheduled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h Successfully assigned openshift-ingress/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h to ip-10-0-133-118.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-a-d74808-5ff59bf748-5cpkf Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d74808-5ff59bf748-5cpkf to ip-10-0-131-200.ec2.internal 13m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 to ip-10-0-131-200.ec2.internal 51m Normal Scheduled pod/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t Successfully assigned openshift-ingress/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t to ip-10-0-133-118.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r to ip-10-0-133-118.ec2.internal 64m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf to ip-10-0-140-105.ec2.internal 47m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj to ip-10-0-133-118.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-b9c4b4-76948497d-7vhrd Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-b9c4b4-76948497d-7vhrd to ip-10-0-140-105.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 to ip-10-0-140-105.ec2.internal 71m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 to ip-10-0-133-118.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-7942f Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-7942f to ip-10-0-133-118.ec2.internal 33m Normal Scheduled pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 Successfully assigned openshift-ingress/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 to ip-10-0-131-200.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-api-b-1c4b82-55c456d68d-4gnsj Successfully assigned openshift-ingress/payload-processing-e2e-api-b-1c4b82-55c456d68d-4gnsj to ip-10-0-131-200.ec2.internal 38m Normal Scheduled pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc Successfully assigned openshift-ingress/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc to ip-10-0-133-118.ec2.internal 47m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn to ip-10-0-140-105.ec2.internal 55m Normal Scheduled pod/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq Successfully assigned openshift-ingress/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq to ip-10-0-133-118.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-api-b-2a2835-c75b7cc9d-shj8b Successfully assigned openshift-ingress/payload-processing-e2e-api-b-2a2835-c75b7cc9d-shj8b to ip-10-0-131-200.ec2.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg to ip-10-0-133-118.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-api-b-aab379-5c6c8cf798-x5xnl Successfully assigned openshift-ingress/payload-processing-e2e-api-b-aab379-5c6c8cf798-x5xnl to ip-10-0-133-118.ec2.internal 34m Normal Scheduled pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn Successfully assigned openshift-ingress/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn to ip-10-0-133-118.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-f2lwf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-f2lwf to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd to ip-10-0-131-200.ec2.internal 47m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm to ip-10-0-140-105.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-api-b-b5ea88-59d675d48b-pwrj9 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-b5ea88-59d675d48b-pwrj9 to ip-10-0-133-118.ec2.internal 29m Normal Scheduled pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Successfully assigned openshift-ingress/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 to ip-10-0-140-105.ec2.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-6gclk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6gclk to ip-10-0-133-118.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd to ip-10-0-133-118.ec2.internal 45m Normal Scheduled pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh Successfully assigned openshift-ingress/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh to ip-10-0-140-105.ec2.internal 37m Normal Scheduled pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Successfully assigned openshift-ingress/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 to ip-10-0-133-118.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-ipp-a-d3a829-698854d99d-8f69p Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-d3a829-698854d99d-8f69p to ip-10-0-140-105.ec2.internal 31m Normal Scheduled pod/payload-processing-e2e-ipp-a-a1854b-6bd77d5846-984lk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a1854b-6bd77d5846-984lk to ip-10-0-140-105.ec2.internal 48m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 to ip-10-0-140-105.ec2.internal 28m Normal Scheduled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp Successfully assigned openshift-ingress/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp to ip-10-0-140-105.ec2.internal 42m Normal Scheduled pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt Successfully assigned openshift-ingress/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt to ip-10-0-131-200.ec2.internal 65m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc to ip-10-0-133-118.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k to ip-10-0-133-118.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-ftnvk Successfully assigned openshift-ingress/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-ftnvk to ip-10-0-140-105.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-a1854b-58595bff5b-vvvvd Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a1854b-58595bff5b-vvvvd to ip-10-0-133-118.ec2.internal 48m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 to ip-10-0-133-118.ec2.internal 31m Normal Scheduled pod/payload-processing-e2e-ipp-a-6176c3-749f86c99b-wfgdk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-6176c3-749f86c99b-wfgdk to ip-10-0-140-105.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-6176c3-65cd57864b-tqgc2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-6176c3-65cd57864b-tqgc2 to ip-10-0-133-118.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-a-33c209-6f4597cc68-xwdkx Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-33c209-6f4597cc68-xwdkx to ip-10-0-140-105.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-33c209-59b7c984c4-9wlm7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-33c209-59b7c984c4-9wlm7 to ip-10-0-133-118.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-wbrs7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-wbrs7 to ip-10-0-140-105.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-2927fd-58947b47f4-wh6th Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2927fd-58947b47f4-wh6th to ip-10-0-140-105.ec2.internal 26m Normal Scheduled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh Successfully assigned openshift-ingress/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh to ip-10-0-133-118.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5842c to ip-10-0-140-105.ec2.internal 63m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 to ip-10-0-140-105.ec2.internal 47m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 to ip-10-0-133-118.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 Successfully assigned openshift-ingress/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 to ip-10-0-133-118.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Successfully assigned openshift-ingress/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v to ip-10-0-133-118.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl to ip-10-0-131-200.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm Successfully assigned openshift-ingress/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm to ip-10-0-131-200.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz to ip-10-0-131-200.ec2.internal 63m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 to ip-10-0-131-200.ec2.internal 57m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc to ip-10-0-133-118.ec2.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 to ip-10-0-140-105.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-k76fk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-k76fk to ip-10-0-133-118.ec2.internal 47m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf to ip-10-0-133-118.ec2.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn to ip-10-0-131-200.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g to ip-10-0-133-118.ec2.internal 20m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-52g42 to ip-10-0-133-118.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c Successfully assigned openshift-ingress/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c to ip-10-0-133-118.ec2.internal 65m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd to ip-10-0-131-200.ec2.internal 57m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t to ip-10-0-133-118.ec2.internal 56m Normal Scheduled pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Successfully assigned openshift-ingress/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg to ip-10-0-140-105.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-409364-openshift-default-868c776847-ntjrp Successfully assigned openshift-ingress/e2e-inf-a-409364-openshift-default-868c776847-ntjrp to ip-10-0-133-118.ec2.internal 54m Normal Scheduled pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 Successfully assigned openshift-ingress/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 to ip-10-0-133-118.ec2.internal 49m Normal Scheduled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Successfully assigned openshift-ingress/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg to ip-10-0-133-118.ec2.internal 43m Normal Scheduled pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t Successfully assigned openshift-ingress/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t to ip-10-0-131-200.ec2.internal 38m Normal Scheduled pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr Successfully assigned openshift-ingress/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr to ip-10-0-140-105.ec2.internal 75m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-98f7774f-vb9c4 Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-98f7774f-vb9c4 to ip-10-0-140-105.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv to ip-10-0-133-118.ec2.internal 56m Normal Scheduled pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g Successfully assigned openshift-ingress/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g to ip-10-0-133-118.ec2.internal 65m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw to ip-10-0-133-118.ec2.internal 75m Normal Scheduled pod/kube-auth-proxy-8996685d-q52jz Successfully assigned openshift-ingress/kube-auth-proxy-8996685d-q52jz to ip-10-0-133-118.ec2.internal 62m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv to ip-10-0-133-118.ec2.internal 38m Normal Scheduled pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 Successfully assigned openshift-ingress/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 to ip-10-0-140-105.ec2.internal 43m Normal Scheduled pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Successfully assigned openshift-ingress/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b to ip-10-0-133-118.ec2.internal 55m Normal Scheduled pod/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc Successfully assigned openshift-ingress/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc to ip-10-0-133-118.ec2.internal 75m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-n49t2 Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-n49t2 to ip-10-0-133-118.ec2.internal 75m Normal Scheduled pod/kube-auth-proxy-8996685d-dlgtt Successfully assigned openshift-ingress/kube-auth-proxy-8996685d-dlgtt to ip-10-0-131-200.ec2.internal 48m Normal Scheduled pod/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr Successfully assigned openshift-ingress/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr to ip-10-0-133-118.ec2.internal 30m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-68bcg Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-68bcg to ip-10-0-131-200.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs to ip-10-0-131-200.ec2.internal 74m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-hdsbg Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hdsbg to ip-10-0-133-118.ec2.internal 44m Normal Scheduled pod/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc Successfully assigned openshift-ingress/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc to ip-10-0-131-200.ec2.internal 62m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk to ip-10-0-140-105.ec2.internal 59m Normal Scheduled pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Successfully assigned openshift-ingress/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 to ip-10-0-133-118.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 to ip-10-0-131-200.ec2.internal 64m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml to ip-10-0-140-105.ec2.internal 65m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn to ip-10-0-133-118.ec2.internal 56m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wjqps to ip-10-0-140-105.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-r5frp Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-r5frp to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc to ip-10-0-131-200.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-p2hhp Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-p2hhp to ip-10-0-140-105.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz Successfully assigned openshift-ingress/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz to ip-10-0-140-105.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-a-409364-58bb5dbb9b-rdzd5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-409364-58bb5dbb9b-rdzd5 to ip-10-0-140-105.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-r4bft Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r4bft to ip-10-0-131-200.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-pkcxp Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-pkcxp to ip-10-0-140-105.ec2.internal 59m Normal Scheduled pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Successfully assigned openshift-ingress/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q to ip-10-0-133-118.ec2.internal 66m Normal Scheduled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc Successfully assigned openshift-ingress/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc to ip-10-0-131-200.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 Successfully assigned openshift-ingress/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 to ip-10-0-133-118.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 to ip-10-0-140-105.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Successfully assigned openshift-ingress/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc to ip-10-0-140-105.ec2.internal 57m Normal Scheduled pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 Successfully assigned openshift-ingress/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 to ip-10-0-133-118.ec2.internal 64m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl to ip-10-0-133-118.ec2.internal 62m Normal Scheduled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Successfully assigned openshift-ingress/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv to ip-10-0-131-200.ec2.internal 32m Normal Scheduled pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Successfully assigned openshift-ingress/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m to ip-10-0-131-200.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx to ip-10-0-131-200.ec2.internal 87m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-659879fb99 from 0 to 1 86m Normal SuccessfulCreate replicaset/router-default-659879fb99 Created pod: router-default-659879fb99-pjvcq 86m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 86m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 86m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 82m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 82m Warning FailedMount pod/router-default-659879fb99-pjvcq MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 82m Warning FailedMount pod/router-default-659879fb99-pjvcq MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 82m Warning FailedMount pod/router-default-659879fb99-pjvcq MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 82m Normal Created pod/router-default-659879fb99-pjvcq Container created 82m Normal Started pod/router-default-659879fb99-pjvcq Container started 82m Normal Pulled pod/router-default-659879fb99-pjvcq Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 82m Normal AddedInterface pod/router-default-659879fb99-pjvcq Add eth0 [10.132.0.19/23] from ovn-kubernetes 75m Normal SuccessfulCreate replicaset/kube-auth-proxy-8996685d Created pod: kube-auth-proxy-8996685d-dlgtt 75m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-8996685d from 0 to 2 75m Normal SuccessfulCreate replicaset/kube-auth-proxy-8996685d Created pod: kube-auth-proxy-8996685d-q52jz 75m Normal AddedInterface pod/kube-auth-proxy-8996685d-dlgtt Add eth0 [10.134.0.21/23] from ovn-kubernetes 75m Normal AddedInterface pod/kube-auth-proxy-8996685d-q52jz Add eth0 [10.133.0.30/23] from ovn-kubernetes 75m Normal Pulling pod/kube-auth-proxy-8996685d-q52jz Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 75m Normal Pulling pod/kube-auth-proxy-8996685d-dlgtt Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 75m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 75m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-n49t2 75m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-n49t2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 75m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-n49t2 Add eth0 [10.133.0.32/23] from ovn-kubernetes 75m Normal Pulled pod/kube-auth-proxy-8996685d-dlgtt Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.131s (2.131s including waiting). Image size: 188904677 bytes. 75m Normal Created pod/kube-auth-proxy-8996685d-dlgtt Container created 75m Normal Started pod/kube-auth-proxy-8996685d-dlgtt Container started 75m Normal Created pod/kube-auth-proxy-8996685d-q52jz Container created 75m Normal Pulled pod/kube-auth-proxy-8996685d-q52jz Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.634s (2.634s including waiting). Image size: 188904677 bytes. 75m Normal Started pod/kube-auth-proxy-8996685d-q52jz Container started 75m Normal Created pod/istiod-openshift-gateway-7876bfcb74-n49t2 Container created 75m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-n49t2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.496s (2.496s including waiting). Image size: 185476477 bytes. 75m Normal Started pod/istiod-openshift-gateway-7876bfcb74-n49t2 Container started 75m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-n49t2 Readiness probe error: HTTP probe failed with statuscode: 503... 75m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-n49t2 Readiness probe failed: HTTP probe failed with statuscode: 503 75m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 75m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 75m Normal Pulling pod/data-science-gateway-data-science-gateway-class-98f7774f-vb9c4 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 75m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-98f7774f-vb9c4 Add eth0 [10.132.0.30/23] from ovn-kubernetes 75m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-98f7774f from 0 to 1 75m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-98f7774f Created pod: data-science-gateway-data-science-gateway-class-98f7774f-vb9c4 75m Normal Created pod/data-science-gateway-data-science-gateway-class-98f7774f-vb9c4 Container created 75m Normal Started pod/data-science-gateway-data-science-gateway-class-98f7774f-vb9c4 Container started 75m Normal Pulled pod/data-science-gateway-data-science-gateway-class-98f7774f-vb9c4 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.451s (2.451s including waiting). Image size: 183555333 bytes. 74m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 74m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 74m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 74m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 74m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 74m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-hdsbg Container started 74m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-hdsbg Container created 74m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-hdsbg Add eth0 [10.133.0.33/23] from ovn-kubernetes 74m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-hdsbg 74m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-hdsbg Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 74m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-hdsbg Readiness probe error: Get "http://10.133.0.33:8080/ready": dial tcp 10.133.0.33:8080: connect: connection refused... 74m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-hdsbg Readiness probe failed: Get "http://10.133.0.33:8080/ready": dial tcp 10.133.0.33:8080: connect: connection refused 74m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-n49t2 74m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-n49t2 Stopping container discovery 74m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 74m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 74m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 73m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 73m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 71m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Add eth0 [10.133.0.44/23] from ovn-kubernetes 71m Normal Pulling pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 71m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 0 to 1 71m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-94bz7 71m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.581s (2.581s including waiting). Image size: 183555333 bytes. 69m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Container started 69m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 69m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Container created 69m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused... 69m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused 69m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-6cf72 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-wstfm 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-5842c 68m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Container created 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Add eth0 [10.133.0.51/23] from ovn-kubernetes 68m Normal Pulling pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Add eth0 [10.134.0.38/23] from ovn-kubernetes 68m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Container started 68m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Container created 68m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Add eth0 [10.132.0.39/23] from ovn-kubernetes 68m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Container started 68m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.004s (3.004s including waiting). Image size: 183555333 bytes. 68m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Container created 68m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Container started 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Startup probe failed: Get "http://10.132.0.39:15021/healthz/ready": dial tcp 10.132.0.39:15021: connect: connection refused 68m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Startup probe error: Get "http://10.132.0.39:15021/healthz/ready": dial tcp 10.132.0.39:15021: connect: connection refused... 68m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-g99dv 68m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-9dw4d 68m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 68m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 68m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 68m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 68m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 68m Normal AddedInterface pod/payload-processing-5f78687d5b-9dw4d Add eth0 [10.133.0.54/23] from ovn-kubernetes 68m Normal Pulling pod/payload-pre-processing-7b98bb678c-g99dv Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 68m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-g99dv Add eth0 [10.133.0.53/23] from ovn-kubernetes 68m Normal Pulling pod/payload-processing-5f78687d5b-9dw4d Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 68m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Startup probe error: Get "http://10.132.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Startup probe failed: Get "http://10.132.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 68m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 68m Normal Started pod/payload-processing-5f78687d5b-9dw4d Container started 68m Normal Created pod/payload-pre-processing-7b98bb678c-g99dv Container created 68m Normal Pulled pod/payload-processing-5f78687d5b-9dw4d Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.399s (2.399s including waiting). Image size: 125317101 bytes. 68m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 68m Normal Pulled pod/payload-pre-processing-7b98bb678c-g99dv Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.524s (2.524s including waiting). Image size: 125317101 bytes. 68m Normal Created pod/payload-processing-5f78687d5b-9dw4d Container created 68m Normal Started pod/payload-pre-processing-7b98bb678c-g99dv Container started 67m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 67m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 4 to 6 67m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-mqc7s 67m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Add eth0 [10.134.0.40/23] from ovn-kubernetes 67m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Container started 67m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Container created 67m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-hszvg 67m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 67m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Add eth0 [10.132.0.40/23] from ovn-kubernetes 67m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Container started 67m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Container created 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 67m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 67m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused 66m Normal AddedLabel gateway/e2e-worker-w1-ca229c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-ca229c 66m Normal AddedLabel gateway/e2e-worker-w5-a7bb0c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-a7bb0c 66m Normal AddedLabel gateway/e2e-worker-w4-46b13b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-46b13b 66m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc 66m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled up replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 0 to 1 66m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled up replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 0 to 1 66m Normal AddedLabel gateway/e2e-worker-w2-da78c4 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-da78c4 66m Normal AddedLabel gateway/e2e-worker-w0-2df0ab Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-2df0ab 66m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled up replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 0 to 1 66m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled up replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 0 to 1 66m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 66m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md 66m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled up replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 0 to 1 66m Normal AddedLabel gateway/e2e-worker-w3-00926c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-00926c 66m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t 66m Normal AddedLabel gateway/e2e-worker-w6-39529e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-39529e 66m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md Add eth0 [10.133.0.56/23] from ovn-kubernetes 66m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Add eth0 [10.133.0.58/23] from ovn-kubernetes 66m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Add eth0 [10.134.0.41/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc 66m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled up replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 0 to 1 66m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Add eth0 [10.133.0.57/23] from ovn-kubernetes 66m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc Add eth0 [10.134.0.42/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj 66m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Add eth0 [10.133.0.59/23] from ovn-kubernetes 66m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled up replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 0 to 1 66m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb 66m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Add eth0 [10.134.0.43/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-00926c-88774844d Created pod: payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-00926c-77f9968595 Error creating: pods "payload-pre-processing-e2e-worker-w3-00926c-77f9968595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-00926c: serviceaccount "payload-processing-e2e-worker-w3-00926c" not found 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-00926c Scaled up replica set payload-pre-processing-e2e-worker-w3-00926c-77f9968595 from 0 to 1 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-00926c-88774844d Error creating: pods "payload-processing-e2e-worker-w3-00926c-88774844d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-00926c: serviceaccount "payload-processing-e2e-worker-w3-00926c" not found 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-00926c-77f9968595 Created pod: payload-pre-processing-e2e-worker-w3-00926c-77f9968595-6zj5z 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-00926c Scaled up replica set payload-processing-e2e-worker-w3-00926c-88774844d from 0 to 1 66m Normal Started pod/payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp Container started 66m Normal AddedInterface pod/payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp Add eth0 [10.133.0.61/23] from ovn-kubernetes 66m Normal Created pod/payload-pre-processing-e2e-worker-w3-00926c-77f9968595-6zj5z Container created 66m Normal Started pod/payload-pre-processing-e2e-worker-w3-00926c-77f9968595-6zj5z Container started 66m Normal Pulled pod/payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-00926c-77f9968595-6zj5z Add eth0 [10.133.0.60/23] from ovn-kubernetes 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-00926c-77f9968595-6zj5z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Created pod/payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp Container created 66m Normal Pulling pod/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-39529e Scaled up replica set payload-processing-e2e-worker-w6-39529e-79cf96b4cb from 0 to 1 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-da78c4-5bf784496d Error creating: pods "payload-processing-e2e-worker-w2-da78c4-5bf784496d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-da78c4: serviceaccount "payload-processing-e2e-worker-w2-da78c4" not found 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-da78c4 Scaled up replica set payload-processing-e2e-worker-w2-da78c4-5bf784496d from 0 to 1 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-da78c4 Scaled up replica set payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff from 0 to 1 66m Normal AddedInterface pod/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt Add eth0 [10.134.0.45/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff Created pod: payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff Error creating: pods "payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-da78c4: serviceaccount "payload-processing-e2e-worker-w2-da78c4" not found 66m Normal Pulling pod/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j Add eth0 [10.134.0.44/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-da78c4-5bf784496d Created pod: payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-39529e Scaled up replica set payload-pre-processing-e2e-worker-w6-39529e-7fcb96875 from 0 to 1 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-2df0ab Scaled up replica set payload-processing-e2e-worker-w0-2df0ab-5649f578f6 from 0 to 1 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-39529e-79cf96b4cb Error creating: pods "payload-processing-e2e-worker-w6-39529e-79cf96b4cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-39529e: serviceaccount "payload-processing-e2e-worker-w6-39529e" not found 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875 Error creating: pods "payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-39529e: serviceaccount "payload-processing-e2e-worker-w6-39529e" not found 66m Normal AddedInterface pod/payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 Add eth0 [10.134.0.47/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875 Created pod: payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z 66m Normal Pulling pod/payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-2df0ab-5649f578f6 Error creating: pods "payload-processing-e2e-worker-w0-2df0ab-5649f578f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2df0ab: serviceaccount "payload-processing-e2e-worker-w0-2df0ab" not found 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f Error creating: pods "payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2df0ab: serviceaccount "payload-processing-e2e-worker-w0-2df0ab" not found 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-39529e-79cf96b4cb Created pod: payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-2df0ab Scaled up replica set payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f from 0 to 1 66m Normal Pulling pod/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z Add eth0 [10.134.0.46/23] from ovn-kubernetes 66m Normal Pulling pod/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal AddedInterface pod/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 Add eth0 [10.134.0.48/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f Created pod: payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Started pod/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj Container started 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-2df0ab-5649f578f6 Created pod: payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 66m Normal Created pod/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj Container created 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj Add eth0 [10.133.0.62/23] from ovn-kubernetes 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f Error creating: pods "payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-a7bb0c: serviceaccount "payload-processing-e2e-worker-w5-a7bb0c" not found 66m Normal Pulled pod/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Pulled pod/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.434s (3.434s including waiting). Image size: 125317101 bytes. 66m Normal Created pod/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j Container created 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.511s (3.511s including waiting). Image size: 125317101 bytes. 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d Created pod: payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-a7bb0c Scaled up replica set payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d from 0 to 1 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d Error creating: pods "payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-a7bb0c: serviceaccount "payload-processing-e2e-worker-w5-a7bb0c" not found 66m Normal Started pod/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 Container started 66m Normal Created pod/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 Container created 66m Normal Pulled pod/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.87s (1.87s including waiting). Image size: 125317101 bytes. 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-a7bb0c Scaled up replica set payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f from 0 to 1 66m Normal Created pod/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt Container created 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f Created pod: payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 66m Normal Started pod/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j Container started 66m Normal Started pod/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt Container started 66m Normal AddedInterface pod/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj Add eth0 [10.133.0.64/23] from ovn-kubernetes 66m Normal Started pod/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 Container started 66m Normal Pulled pod/payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.132s (3.132s including waiting). Image size: 125317101 bytes. 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-46b13b-646fb546c8 Error creating: pods "payload-processing-e2e-worker-w4-46b13b-646fb546c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-46b13b: serviceaccount "payload-processing-e2e-worker-w4-46b13b" not found 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-46b13b-646fb546c8 Created pod: payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 66m Normal Started pod/payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 Container started 66m Normal Created pod/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj Container created 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc Error creating: pods "payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-46b13b: serviceaccount "payload-processing-e2e-worker-w4-46b13b" not found 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc Created pod: payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn 66m Normal Created pod/payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 Container created 66m Normal Started pod/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj Container started 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.249s (3.249s including waiting). Image size: 125317101 bytes. 66m Normal Created pod/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z Container created 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-46b13b Scaled up replica set payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc from 0 to 1 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-46b13b Scaled up replica set payload-processing-e2e-worker-w4-46b13b-646fb546c8 from 0 to 1 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 Add eth0 [10.134.0.49/23] from ovn-kubernetes 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Started pod/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z Container started 66m Normal Created pod/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 Container created 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Started pod/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn Container started 66m Normal Pulled pod/payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal AddedInterface pod/payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 Add eth0 [10.133.0.65/23] from ovn-kubernetes 66m Normal Created pod/payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 Container created 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn Add eth0 [10.134.0.50/23] from ovn-kubernetes 66m Normal Started pod/payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 Container started 66m Normal Created pod/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn Container created 66m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md Container created 66m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md Container started 66m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-ca229c Scaled up replica set payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6 from 0 to 1 66m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5 Error creating: pods "payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-ca229c: serviceaccount "payload-processing-e2e-worker-w1-ca229c" not found 66m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6 Error creating: pods "payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-ca229c: serviceaccount "payload-processing-e2e-worker-w1-ca229c" not found 66m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-ca229c Scaled up replica set payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5 from 0 to 1 66m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5 Created pod: payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz 66m Normal Created pod/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf Container created 66m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz Add eth0 [10.134.0.52/23] from ovn-kubernetes 66m Normal AddedInterface pod/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf Add eth0 [10.134.0.53/23] from ovn-kubernetes 66m Normal Pulled pod/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Normal Started pod/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz Container started 66m Normal Started pod/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf Container started 66m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6 Created pod: payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf 66m Normal Created pod/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz Container created 66m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 66m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled up replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 1 to 4 66m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v Add eth0 [10.132.0.44/23] from ovn-kubernetes 66m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Container started 66m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc Container started 66m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd 66m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Container started 66m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc Container created 66m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v 66m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Container started 66m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn 66m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v Container created 66m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Container created 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Container created 66m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Container started 66m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Add eth0 [10.133.0.67/23] from ovn-kubernetes 66m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Container created 66m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v Container started 66m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Container created 66m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Add eth0 [10.134.0.54/23] from ovn-kubernetes 66m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Readiness probe failed: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 66m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Readiness probe error: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 66m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Container started 66m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Container created 66m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v Startup probe failed: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused 66m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 66m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 66m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v Startup probe error: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 66m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 66m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Container started 66m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Container created 66m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled up replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 1 to 2 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Add eth0 [10.134.0.56/23] from ovn-kubernetes 65m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Container started 65m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Container created 65m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Add eth0 [10.133.0.69/23] from ovn-kubernetes 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled up replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 1 to 2 65m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Container started 65m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Container created 65m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 65m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 65m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 65m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Container created 65m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Container started 65m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 65m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled up replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 1 to 2 65m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled up replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 1 to 2 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 65m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Container started 65m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b 65m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Add eth0 [10.132.0.50/23] from ovn-kubernetes 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled up replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 4 to 8 65m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Container created 65m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled up replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 1 to 2 65m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Container started 65m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Container created 65m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Add eth0 [10.133.0.70/23] from ovn-kubernetes 65m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Add eth0 [10.134.0.59/23] from ovn-kubernetes 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default New size: 8; reason: 65m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Container started 65m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Container created 65m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n 65m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Container created 65m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Add eth0 [10.133.0.71/23] from ovn-kubernetes 65m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Container created 65m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Container started 65m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Container started 65m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Add eth0 [10.134.0.60/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Add eth0 [10.133.0.72/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd 65m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 65m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc 65m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Container started 65m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Container created 65m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Container started 65m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Container created 65m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Add eth0 [10.134.0.61/23] from ovn-kubernetes 65m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 65m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 65m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 65m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 65m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 65m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused... 65m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 65m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 65m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 65m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 65m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 65m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Container started 65m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Container created 65m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled up replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 8 to 10 65m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Add eth0 [10.133.0.73/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Add eth0 [10.132.0.52/23] from ovn-kubernetes 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default New size: 10; reason: All metrics below target 65m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Container created 65m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Container started 65m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused... 65m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 65m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 64m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled up replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 2 to 4 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 64m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Container started 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 64m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Add eth0 [10.132.0.55/23] from ovn-kubernetes 64m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 64m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg 64m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled up replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 2 to 4 64m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Add eth0 [10.133.0.75/23] from ovn-kubernetes 64m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Container started 64m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Container created 64m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml 64m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Container created 64m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl 64m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Container created 64m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Container started 64m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Container started 64m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Add eth0 [10.133.0.76/23] from ovn-kubernetes 64m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Add eth0 [10.132.0.56/23] from ovn-kubernetes 64m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Container created 64m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused... 64m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 64m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 64m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 64m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 64m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Container created 64m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Container started 64m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled up replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 2 to 3 64m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 64m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls 64m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled up replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 2 to 3 64m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Add eth0 [10.132.0.57/23] from ovn-kubernetes 64m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Add eth0 [10.132.0.58/23] from ovn-kubernetes 64m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Container created 64m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Container started 64m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused 64m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused... 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 64m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled up replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 2 to 3 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 64m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Container started 64m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Container created 64m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled up replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 1 to 2 64m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf 64m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Add eth0 [10.132.0.59/23] from ovn-kubernetes 64m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk 64m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk Add eth0 [10.134.0.65/23] from ovn-kubernetes 64m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk Container created 64m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk Container started 64m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Readiness probe error: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 63m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Readiness probe failed: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 63m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Add eth0 [10.133.0.83/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-tfw9l 63m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Container created 63m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Container started 63m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 6 to 7 63m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z 63m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled up replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 3 to 4 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Add eth0 [10.133.0.85/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Container created 63m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Container started 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 63m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 63m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-ww87s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 63m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Container created 63m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx Container started 63m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-ww87s 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-t5tsx 63m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 7 to 10 63m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-ww87s Add eth0 [10.133.0.86/23] from ovn-kubernetes 63m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 63m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx Container created 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Add eth0 [10.132.0.60/23] from ovn-kubernetes 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx Add eth0 [10.134.0.70/23] from ovn-kubernetes 63m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Container started 63m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-ww87s Container started 63m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-ww87s Container created 63m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Add eth0 [10.134.0.71/23] from ovn-kubernetes 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx 63m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled up replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 2 to 4 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp 63m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 63m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Add eth0 [10.132.0.61/23] from ovn-kubernetes 63m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled up replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 4 to 5 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled up replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 4 to 6 63m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Add eth0 [10.132.0.62/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr 63m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Add eth0 [10.134.0.72/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Container created 63m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Container created 63m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Add eth0 [10.133.0.87/23] from ovn-kubernetes 63m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Container created 63m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Container started 63m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Container created 63m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Container created 63m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Container started 63m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Container started 63m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Container started 63m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Container started 63m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-ww87s Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 63m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused... 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 63m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-ww87s Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 63m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 63m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 63m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 63m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 63m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k 63m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 63m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled up replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 3 to 6 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled up replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 5 to 7 63m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Add eth0 [10.132.0.64/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Container created 63m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Container started 63m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Container created 63m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Container started 63m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Add eth0 [10.134.0.75/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Container started 63m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Container created 63m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Add eth0 [10.132.0.63/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Container started 63m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Container created 63m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Add eth0 [10.133.0.89/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Container started 63m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Container created 63m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Add eth0 [10.133.0.88/23] from ovn-kubernetes 63m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled up replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 3 to 4 63m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 63m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Add eth0 [10.133.0.90/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Container started 63m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Container created 63m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 62m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled up replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 4 to 8 62m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Container started 62m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Container started 62m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Container created 62m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Container created 62m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Add eth0 [10.134.0.79/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Add eth0 [10.133.0.92/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Add eth0 [10.132.0.65/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Container started 62m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Container created 62m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Container created 62m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Add eth0 [10.134.0.78/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Container started 62m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Startup probe failed: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Startup probe error: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 10; reason: 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf (combined from similar events): Created pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 62m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled up replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 8 to 10 62m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Add eth0 [10.132.0.66/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Container created 62m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Container started 62m Normal Started pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Container started 62m Normal AddedInterface pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Add eth0 [10.133.0.94/23] from ovn-kubernetes 62m Normal Created pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Container created 62m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused... 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f (combined from similar events): Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk 62m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv 62m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled up replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 7 to 10 62m Normal SuccessfulCreate replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Created pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv 62m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Container created 62m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Container started 62m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Add eth0 [10.132.0.67/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Container created 62m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Container started 62m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Add eth0 [10.134.0.81/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Add eth0 [10.133.0.96/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Container created 62m Normal Started pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Container started 62m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr 61m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Container started 61m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Add eth0 [10.134.0.83/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Container created 61m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Container started 61m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Add eth0 [10.133.0.101/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc 61m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Add eth0 [10.133.0.100/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c (combined from similar events): Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv 61m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Container created 61m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Created pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs 61m Normal AddedInterface pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Add eth0 [10.132.0.69/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Container created 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled up replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 6 to 10 61m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Container started 61m Normal Created pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Container created 61m Normal Pulled pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Container started 61m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Startup probe error: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Startup probe failed: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Startup probe error: Get "http://10.133.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Startup probe failed: Get "http://10.133.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled up replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 6 to 7 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq 61m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq Container created 61m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq Add eth0 [10.133.0.104/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq Container started 61m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq Startup probe failed: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq Startup probe error: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused... 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw 61m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled up replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 4 to 8 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs 61m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled up replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 7 to 9 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 61m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Container created 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 61m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf 61m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs Container started 61m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs Container created 61m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs Add eth0 [10.134.0.90/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw Add eth0 [10.132.0.71/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Add eth0 [10.132.0.72/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled up replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 4 to 7 61m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw Container created 61m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Container started 61m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Container created 61m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw Container started 61m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv Add eth0 [10.134.0.92/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv Container started 61m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Container started 61m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv Container created 61m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Add eth0 [10.134.0.91/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc 61m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c Add eth0 [10.133.0.105/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c Container started 61m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c Container created 61m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Container started 61m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Add eth0 [10.134.0.93/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Add eth0 [10.133.0.106/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc Add eth0 [10.132.0.73/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Container created 61m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Container started 61m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc Container created 61m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc Container started 61m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Container created 61m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld 60m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled up replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 8 to 9 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 9; reason: All metrics below target 60m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Container created 60m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Container started 60m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Add eth0 [10.132.0.74/23] from ovn-kubernetes 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-r4bft 60m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-r4bft Add eth0 [10.134.0.94/23] from ovn-kubernetes 60m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-r4bft Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-r4bft Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.614s (2.614s including waiting). Image size: 185476477 bytes. 60m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-r4bft Container created 60m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-r4bft Container started 60m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused... 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp 60m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled up replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 7 to 10 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 (combined from similar events): Created pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h 60m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff Add eth0 [10.134.0.96/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h Add eth0 [10.133.0.114/23] from ovn-kubernetes 60m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Container created 60m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h Container started 60m Normal AddedInterface pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Add eth0 [10.132.0.76/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff Container started 60m Normal Pulled pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff Container created 60m Normal Started pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Container started 60m Normal Created pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h Container created 60m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Readiness probe error: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Readiness probe failed: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Readiness probe error: Get "http://10.132.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Readiness probe failed: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Readiness probe error: Get "http://10.133.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Readiness probe failed: Get "http://10.133.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Readiness probe error: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Readiness probe failed: Get "http://10.132.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Readiness probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Readiness probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h Startup probe failed: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h Startup probe error: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused... 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled up replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 9 to 10 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 (combined from similar events): Created pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 59m Normal AddedInterface pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 Add eth0 [10.133.0.120/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 Container created 59m Normal Started pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 Container started 59m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Readiness probe failed: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Readiness probe error: Get "http://10.132.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Readiness probe error: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Readiness probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Readiness probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Readiness probe failed: Get "http://10.132.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Readiness probe failed: Get "http://10.133.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Readiness probe error: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Readiness probe failed: Get "http://10.132.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Readiness probe failed: Get "http://10.134.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Readiness probe error: Get "http://10.134.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Readiness probe error: Get "http://10.132.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Readiness probe failed: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Readiness probe error: Get "http://10.133.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Readiness probe failed: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Readiness probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Readiness probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Readiness probe error: Get "http://10.133.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Readiness probe failed: Get "http://10.132.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Readiness probe error: Get "http://10.132.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Readiness probe failed: Get "http://10.132.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Readiness probe error: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Readiness probe error: Get "http://10.132.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Readiness probe error: Get "http://10.132.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Readiness probe failed: Get "http://10.132.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Readiness probe failed: Get "http://10.132.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Readiness probe failed: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Readiness probe failed: Get "http://10.133.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Readiness probe failed: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Readiness probe error: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Readiness probe error: Get "http://10.132.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Readiness probe failed: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Readiness probe error: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Readiness probe failed: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Readiness probe error: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Readiness probe error: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Readiness probe failed: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Readiness probe failed: Get "http://10.133.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Readiness probe failed: Get "http://10.133.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Readiness probe error: Get "http://10.133.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Readiness probe error: Get "http://10.133.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 Startup probe failed: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 Startup probe error: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused... 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled up replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 9 to 10 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c (combined from similar events): Created pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 59m Normal Pulled pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Add eth0 [10.133.0.125/23] from ovn-kubernetes 59m Normal Created pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Container created 59m Normal Started pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Container started 59m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Startup probe error: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Startup probe failed: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused 59m Normal SuccessfulCreate replicaset/e2e-migrate-da21809f-gw-openshift-default-654f97494f Created pod: e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 59m Normal AddedLabel gateway/e2e-derive-ea3e7b1d Added label istio.io/rev=openshift-gateway to gateway e2e-derive-ea3e7b1d 59m Normal AddedLabel gateway/e2e-migrate-da21809f-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-da21809f-gw 59m Normal ScalingReplicaSet deployment/e2e-migrate-da21809f-gw-openshift-default Scaled up replica set e2e-migrate-da21809f-gw-openshift-default-654f97494f from 0 to 1 59m Normal AddedInterface pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Add eth0 [10.133.0.128/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-derive-ea3e7b1d-openshift-default Scaled up replica set e2e-derive-ea3e7b1d-openshift-default-65c65b785 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-derive-ea3e7b1d-openshift-default-65c65b785 Created pod: e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q 59m Normal AddedInterface pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Add eth0 [10.133.0.129/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f Created pod: payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-wpppq 58m Warning FailedCreate replicaset/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b Error creating: pods "payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-da21809f: serviceaccount "payload-processing-e2e-migrate-da21809f" not found 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-da21809f Scaled up replica set payload-processing-e2e-migrate-da21809f-7c7c7f9b8b from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-da21809f Scaled up replica set payload-pre-processing-e2e-migrate-da21809f-78d4d5887f from 0 to 1 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b Created pod: payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-kgnb7 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f Error creating: pods "payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-da21809f: serviceaccount "payload-processing-e2e-migrate-da21809f" not found 58m Normal Pulled pod/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-kgnb7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Started pod/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-wpppq Container started 58m Normal Created pod/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-wpppq Container created 58m Normal Pulled pod/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-wpppq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-wpppq Add eth0 [10.133.0.131/23] from ovn-kubernetes 58m Normal AddedInterface pod/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-kgnb7 Add eth0 [10.133.0.132/23] from ovn-kubernetes 58m Normal Created pod/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-kgnb7 Container created 58m Normal Started pod/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-kgnb7 Container started 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-ea3e7b1d Scaled up replica set payload-processing-e2e-derive-ea3e7b1d-77f5ddd568 from 0 to 1 58m Normal AddedInterface pod/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w Add eth0 [10.132.0.77/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568 Created pod: payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 58m Normal Pulling pod/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f Error creating: pods "payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-ea3e7b1d: serviceaccount "payload-processing-e2e-derive-ea3e7b1d" not found 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f Created pod: payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-ea3e7b1d Scaled up replica set payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568 Error creating: pods "payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-ea3e7b1d: serviceaccount "payload-processing-e2e-derive-ea3e7b1d" not found 58m Normal Pulled pod/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 Container created 58m Normal Started pod/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 Container started 58m Normal AddedInterface pod/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 Add eth0 [10.133.0.133/23] from ovn-kubernetes 58m Normal Started pod/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w Container started 58m Normal Created pod/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w Container created 58m Normal Pulled pod/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.766s (4.766s including waiting). Image size: 125317101 bytes. 58m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Readiness probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Readiness probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Normal ScalingReplicaSet deployment/e2e-ait-25917498-openshift-default Scaled up replica set e2e-ait-25917498-openshift-default-77dc9bb847 from 0 to 1 58m Normal AddedLabel gateway/e2e-ait-25917498 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-25917498 58m Normal SuccessfulCreate replicaset/e2e-ait-25917498-openshift-default-77dc9bb847 Created pod: e2e-ait-25917498-openshift-default-77dc9bb847-smvqn 58m Normal AddedInterface pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Add eth0 [10.132.0.78/23] from ovn-kubernetes 58m Warning FailedCreate replicaset/payload-processing-e2e-ait-25917498-5b656767dd Error creating: pods "payload-processing-e2e-ait-25917498-5b656767dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-25917498: serviceaccount "payload-processing-e2e-ait-25917498" not found 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-25917498 Scaled up replica set payload-processing-e2e-ait-25917498-5b656767dd from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-25917498 Scaled up replica set payload-pre-processing-e2e-ait-25917498-769cb747f9 from 0 to 1 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-25917498-769cb747f9 Error creating: pods "payload-pre-processing-e2e-ait-25917498-769cb747f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-25917498: serviceaccount "payload-processing-e2e-ait-25917498" not found 58m Normal AddedInterface pod/payload-pre-processing-e2e-ait-25917498-769cb747f9-hhxfb Add eth0 [10.134.0.104/23] from ovn-kubernetes 58m Normal Pulled pod/payload-pre-processing-e2e-ait-25917498-769cb747f9-hhxfb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-pre-processing-e2e-ait-25917498-769cb747f9-hhxfb Container created 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-25917498-5b656767dd Created pod: payload-processing-e2e-ait-25917498-5b656767dd-2xdq5 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-25917498-769cb747f9 Created pod: payload-pre-processing-e2e-ait-25917498-769cb747f9-hhxfb 58m Normal Started pod/payload-pre-processing-e2e-ait-25917498-769cb747f9-hhxfb Container started 58m Normal Pulled pod/payload-processing-e2e-ait-25917498-5b656767dd-2xdq5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-processing-e2e-ait-25917498-5b656767dd-2xdq5 Container created 58m Normal AddedInterface pod/payload-processing-e2e-ait-25917498-5b656767dd-2xdq5 Add eth0 [10.134.0.105/23] from ovn-kubernetes 58m Normal Started pod/payload-processing-e2e-ait-25917498-5b656767dd-2xdq5 Container started 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-da21809f-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-da21809f-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-ea3e7b1d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-ea3e7b1d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-wstfm 58m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-ww87s 58m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-tfw9l 58m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-t5tsx 58m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Stopping container istio-proxy 58m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-wstfm Stopping container istio-proxy 58m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-94bz7 58m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Stopping container istio-proxy 58m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-t5tsx Stopping container istio-proxy 58m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-ww87s Stopping container istio-proxy 58m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 10 to 5 58m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Readiness probe error: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Readiness probe failed: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Readiness probe error: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Readiness probe failed: HTTP probe failed with statuscode: 503 58m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Readiness probe failed: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-94bz7 Readiness probe error: HTTP probe failed with statuscode: 503... 58m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Readiness probe failed: Get "http://10.134.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Readiness probe error: Get "http://10.134.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd 58m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b 58m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Readiness probe failed: Get "http://10.133.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v 58m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn 58m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc 58m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc 58m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-f2dhc Stopping container istio-proxy 58m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Readiness probe error: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-trq2v Stopping container istio-proxy 58m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled down replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 10 to 4 58m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Readiness probe error: Get "http://10.133.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Stopping container istio-proxy 58m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Stopping container istio-proxy 58m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Stopping container istio-proxy 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Readiness probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-tfw9l Readiness probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 58m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Stopping container istio-proxy 58m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Readiness probe failed: HTTP probe failed with statuscode: 503 58m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Readiness probe failed: HTTP probe failed with statuscode: 503 58m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-ft9tn Readiness probe error: HTTP probe failed with statuscode: 503... 58m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x676b Readiness probe error: HTTP probe failed with statuscode: 503... 58m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Readiness probe error: HTTP probe failed with statuscode: 503... 58m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Readiness probe error: HTTP probe failed with statuscode: 503... 58m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4n9lc Readiness probe failed: HTTP probe failed with statuscode: 503 58m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-4gzzd Readiness probe failed: HTTP probe failed with statuscode: 503 58m Normal Started pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Container started 58m Normal Created pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Container created 58m Normal Pulled pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Container started 58m Normal Pulled pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Container created 58m Warning ProbeError pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-da21809f-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-da21809f-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning ProbeError pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused... 58m Normal Killing pod/payload-pre-processing-e2e-migrate-da21809f-78d4d5887f-wpppq Stopping container payload-pre-processing 58m Normal Killing pod/payload-processing-e2e-migrate-da21809f-7c7c7f9b8b-kgnb7 Stopping container payload-processing 58m Warning Unhealthy pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f (combined from similar events): Created pod: maas-default-gateway-openshift-default-c9ffffc7f-jvvbq 57m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq Container created 57m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-6gclk Container started 57m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Container created 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq Add eth0 [10.134.0.107/23] from ovn-kubernetes 57m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-6gclk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-6gclk Add eth0 [10.133.0.143/23] from ovn-kubernetes 57m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Container started 57m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Container created 57m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Container started 57m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg Container started 57m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq Container started 57m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-6gclk Container created 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg Add eth0 [10.133.0.144/23] from ovn-kubernetes 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Add eth0 [10.133.0.141/23] from ovn-kubernetes 57m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Add eth0 [10.133.0.142/23] from ovn-kubernetes 57m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg Container created 57m Warning Unhealthy pod/payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp Readiness probe failed: timeout: health rpc did not complete within 1s 57m Normal Killing pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Stopping container istio-proxy 57m Normal AddedLabel gateway/e2e-ait-4522b19c Added label istio.io/rev=openshift-gateway to gateway e2e-ait-4522b19c 57m Normal SuccessfulCreate replicaset/e2e-ait-4522b19c-openshift-default-b9d85f877 Created pod: e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 57m Normal Killing pod/payload-pre-processing-e2e-derive-ea3e7b1d-6fb74b859f-kxc7w Stopping container payload-pre-processing 57m Normal ScalingReplicaSet deployment/e2e-ait-4522b19c-openshift-default Scaled up replica set e2e-ait-4522b19c-openshift-default-b9d85f877 from 0 to 1 57m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused 57m Warning ProbeError pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Readiness probe error: HTTP probe failed with statuscode: 503... 57m Warning Unhealthy pod/e2e-migrate-da21809f-gw-openshift-default-654f97494f-h8rc5 Readiness probe failed: HTTP probe failed with statuscode: 503 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-ea3e7b1d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-ea3e7b1d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal Killing pod/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 Stopping container payload-processing 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Readiness probe failed: Get "http://10.132.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Readiness probe error: Get "http://10.132.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Normal AddedLabel gateway/e2e-shared-a-w6-75443e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-75443e 57m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-75443e-openshift-default Scaled up replica set e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc Created pod: e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 57m Normal AddedInterface pod/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 Add eth0 [10.133.0.146/23] from ovn-kubernetes 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Startup probe failed: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused 57m Warning ProbeError pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Readiness probe error: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning Unhealthy pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Readiness probe failed: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-6gclk Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Startup probe error: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused... 57m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 57m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 57m Normal AddedInterface pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 Add eth0 [10.133.0.147/23] from ovn-kubernetes 57m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-6gclk Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 57m Normal Pulled pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Container created 57m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Startup probe error: Get "http://10.133.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Normal Started pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Container started 57m Warning Unhealthy pod/payload-processing-e2e-derive-ea3e7b1d-77f5ddd568-r9gb2 Readiness probe failed: timeout: failed to connect service "10.133.0.133:50052" within 1s: context deadline exceeded 57m Warning FailedCreate replicaset/payload-processing-e2e-ait-4522b19c-b46b9b49d Error creating: pods "payload-processing-e2e-ait-4522b19c-b46b9b49d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4522b19c: serviceaccount "payload-processing-e2e-ait-4522b19c" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9 Created pod: payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-9w8xf 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-4522b19c-b46b9b49d Created pod: payload-processing-e2e-ait-4522b19c-b46b9b49d-4ddlh 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9 Error creating: pods "payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4522b19c: serviceaccount "payload-processing-e2e-ait-4522b19c" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-4522b19c Scaled up replica set payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9 from 0 to 1 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-4522b19c Scaled up replica set payload-processing-e2e-ait-4522b19c-b46b9b49d from 0 to 1 57m Normal Started pod/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-9w8xf Container started 57m Normal AddedInterface pod/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-9w8xf Add eth0 [10.134.0.108/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-9w8xf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-9w8xf Container created 57m Normal Started pod/payload-processing-e2e-ait-4522b19c-b46b9b49d-4ddlh Container started 57m Normal AddedInterface pod/payload-processing-e2e-ait-4522b19c-b46b9b49d-4ddlh Add eth0 [10.134.0.109/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-ait-4522b19c-b46b9b49d-4ddlh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-ait-4522b19c-b46b9b49d-4ddlh Container created 57m Warning ProbeError pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 57m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled up replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 4 to 7 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 (combined from similar events): Created pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc 57m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc Container started 57m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t Add eth0 [10.133.0.148/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t Container created 57m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 Container started 57m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 Add eth0 [10.133.0.149/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 Container created 57m Normal Created pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc Container created 57m Normal Started pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t Container started 57m Normal Pulled pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc Add eth0 [10.133.0.150/23] from ovn-kubernetes 57m Normal Killing pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Stopping container istio-proxy 57m Warning Unhealthy pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Readiness probe failed: HTTP probe failed with statuscode: 503 57m Warning ProbeError pod/e2e-derive-ea3e7b1d-openshift-default-65c65b785-8tt9q Readiness probe error: HTTP probe failed with statuscode: 503... 57m Normal Killing pod/payload-pre-processing-e2e-ait-25917498-769cb747f9-hhxfb Stopping container payload-pre-processing 57m Normal Killing pod/payload-processing-e2e-ait-25917498-5b656767dd-2xdq5 Stopping container payload-processing 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc Created pod: payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-75443e Scaled up replica set payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc from 0 to 1 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-75443e Scaled up replica set payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc Error creating: pods "payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-75443e: serviceaccount "payload-processing-e2e-shared-a-w6-75443e" not found 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb Created pod: payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 57m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb Error creating: pods "payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-75443e: serviceaccount "payload-processing-e2e-shared-a-w6-75443e" not found 57m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Started pod/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 Container started 57m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc Container started 57m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc Container created 57m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc Add eth0 [10.132.0.81/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 Add eth0 [10.132.0.82/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 Container created 57m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc Startup probe error: Get "http://10.133.0.150:15021/healthz/ready": dial tcp 10.133.0.150:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc Startup probe failed: Get "http://10.133.0.150:15021/healthz/ready": dial tcp 10.133.0.150:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t Startup probe error: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 Startup probe failed: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 Startup probe error: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t Startup probe failed: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-25917498-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-25917498-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal Killing pod/e2e-ait-25917498-openshift-default-77dc9bb847-smvqn Stopping container istio-proxy 56m Normal AddedLabel gateway/e2e-mt-9dba4af9 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-9dba4af9 56m Normal SuccessfulCreate replicaset/e2e-mt-9dba4af9-openshift-default-7d7d889b5 Created pod: e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg 56m Normal ScalingReplicaSet deployment/e2e-mt-9dba4af9-openshift-default Scaled up replica set e2e-mt-9dba4af9-openshift-default-7d7d889b5 from 0 to 1 56m Normal AddedInterface pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Add eth0 [10.132.0.83/23] from ovn-kubernetes 56m Normal Started pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 Container started 56m Normal Pulled pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 Container created 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47 Error creating: pods "payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9dba4af9: serviceaccount "payload-processing-e2e-mt-9dba4af9" not found 56m Warning FailedCreate replicaset/payload-processing-e2e-mt-9dba4af9-6d98fd4964 Error creating: pods "payload-processing-e2e-mt-9dba4af9-6d98fd4964-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9dba4af9: serviceaccount "payload-processing-e2e-mt-9dba4af9" not found 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-9dba4af9 Scaled up replica set payload-processing-e2e-mt-9dba4af9-6d98fd4964 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47 Created pod: payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl 56m Normal Started pod/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 Container started 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-9dba4af9 Scaled up replica set payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47 from 0 to 1 56m Normal Created pod/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 Container created 56m Normal Pulled pod/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl Add eth0 [10.134.0.112/23] from ovn-kubernetes 56m Normal Pulled pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr Add eth0 [10.132.0.84/23] from ovn-kubernetes 56m Normal Started pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl Container started 56m Normal Created pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr Container created 56m Normal Pulled pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl Container created 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-9dba4af9-6d98fd4964 Created pod: payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr 56m Normal Killing pod/payload-pre-processing-e2e-ait-4522b19c-57c8cd98c9-9w8xf Stopping container payload-pre-processing 56m Normal Killing pod/payload-processing-e2e-ait-4522b19c-b46b9b49d-4ddlh Stopping container payload-processing 56m Normal AddedInterface pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Add eth0 [10.132.0.85/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99 Created pod: e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b 56m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0efe43-openshift-default Scaled up replica set e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99 from 0 to 1 56m Normal AddedLabel gateway/e2e-shared-b-w6-0efe43 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-0efe43 56m Warning ProbeError pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 Startup probe error: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 Startup probe failed: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused 56m Warning ProbeError pod/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 Startup probe error: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 Startup probe failed: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused 56m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-4522b19c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 Add eth0 [10.134.0.113/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-0efe43 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc from 0 to 1 56m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6 Error creating: pods "payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-0efe43: serviceaccount "payload-processing-e2e-shared-b-w6-0efe43" not found 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6 Created pod: payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-0efe43 Scaled up replica set payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc Created pod: payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc Error creating: pods "payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-0efe43: serviceaccount "payload-processing-e2e-shared-b-w6-0efe43" not found 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-4522b19c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal Started pod/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 Container started 56m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 Container created 56m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 Container started 56m Normal Pulled pod/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 Add eth0 [10.134.0.114/23] from ovn-kubernetes 56m Normal Created pod/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 Container created 56m Normal AddedLabel gateway/e2e-shared-a-w4-e371c3 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-e371c3 56m Normal Killing pod/e2e-ait-4522b19c-openshift-default-b9d85f877-wj464 Stopping container istio-proxy 56m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e371c3-openshift-default Scaled up replica set e2e-shared-a-w4-e371c3-openshift-default-699697d895 from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e371c3-openshift-default-699697d895 Created pod: e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g 56m Normal AddedInterface pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g Add eth0 [10.133.0.154/23] from ovn-kubernetes 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-75443e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-75443e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6 Error creating: pods "payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-e371c3: serviceaccount "payload-processing-e2e-shared-a-w4-e371c3" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc Error creating: pods "payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-e371c3: serviceaccount "payload-processing-e2e-shared-a-w4-e371c3" not found 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6 Created pod: payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc Created pod: payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-e371c3 Scaled up replica set payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6 from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-e371c3 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc from 0 to 1 56m Normal Started pod/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v Container started 56m Normal Pulled pod/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh Add eth0 [10.134.0.115/23] from ovn-kubernetes 56m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v Container created 56m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh Container created 56m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v Add eth0 [10.134.0.116/23] from ovn-kubernetes 56m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 56m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-wjqps 56m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 56m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 56m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Add eth0 [10.132.0.86/23] from ovn-kubernetes 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-75443e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-75443e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.448s (3.448s including waiting). Image size: 185476477 bytes. 56m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Container created 56m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Container started 56m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Readiness probe error: HTTP probe failed with statuscode: 503... 56m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Readiness probe failed: HTTP probe failed with statuscode: 503 55m Warning Unhealthy pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Readiness probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Readiness probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning ProbeError pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Readiness probe error: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Readiness probe failed: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Normal Created pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Container created 55m Normal Pulled pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Container created 55m Normal Started pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Container started 55m Normal Started pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Container started 55m Normal AddedLabel gateway/e2e-shared-a-w3-5704d6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-5704d6 55m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-5704d6-openshift-default Scaled up replica set e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9 from 0 to 1 55m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9 Created pod: e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq 55m Normal AddedInterface pod/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq Add eth0 [10.133.0.161/23] from ovn-kubernetes 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-9dba4af9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-9dba4af9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning Unhealthy pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 55m Warning ProbeError pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-5704d6 Scaled up replica set payload-processing-e2e-shared-a-w3-5704d6-695bc45c86 from 0 to 1 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-5704d6 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899 from 0 to 1 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-0efe43-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-0efe43-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-5704d6: serviceaccount "payload-processing-e2e-shared-a-w3-5704d6" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899 Created pod: payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86 Created pod: payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc 55m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86 Error creating: pods "payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-5704d6: serviceaccount "payload-processing-e2e-shared-a-w3-5704d6" not found 55m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc Container created 55m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp Add eth0 [10.134.0.122/23] from ovn-kubernetes 55m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp Container started 55m Normal Started pod/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc Container started 55m Normal Pulled pod/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp Container created 55m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc Add eth0 [10.134.0.123/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 Created pod: e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h 55m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-a59ecb-openshift-default Scaled up replica set e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 from 0 to 1 55m Normal AddedLabel gateway/e2e-shared-b-w4-a59ecb Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-a59ecb 55m Normal AddedInterface pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Add eth0 [10.133.0.162/23] from ovn-kubernetes 55m Normal Started pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g Container started 55m Normal Created pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g Container created 55m Normal Pulled pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4 Error creating: pods "payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-a59ecb: serviceaccount "payload-processing-e2e-shared-b-w4-a59ecb" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4 Created pod: payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-a59ecb Scaled up replica set payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc Created pod: payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs 55m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs Add eth0 [10.134.0.124/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-a59ecb Scaled up replica set payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc from 0 to 1 55m Normal Pulled pod/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn Container started 55m Normal Created pod/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn Container created 55m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs Container created 55m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs Container started 55m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn Add eth0 [10.134.0.125/23] from ovn-kubernetes 55m Warning ProbeError pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 55m Normal AddedLabel gateway/e2e-mt-01955278 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-01955278 55m Normal ScalingReplicaSet deployment/e2e-mt-01955278-openshift-default Scaled up replica set e2e-mt-01955278-openshift-default-56c64d6797 from 0 to 1 55m Normal SuccessfulCreate replicaset/e2e-mt-01955278-openshift-default-56c64d6797 Created pod: e2e-mt-01955278-openshift-default-56c64d6797-9hfnc 55m Normal AddedInterface pod/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc Add eth0 [10.133.0.166/23] from ovn-kubernetes 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-9dba4af9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-01955278 Scaled up replica set payload-processing-e2e-mt-01955278-7d4f98b5fd from 0 to 1 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-9dba4af9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal Started pod/payload-processing-e2e-mt-01955278-7d4f98b5fd-ltzh8 Container started 55m Normal AddedInterface pod/payload-pre-processing-e2e-mt-01955278-f55689567-97lbq Add eth0 [10.134.0.128/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-processing-e2e-mt-01955278-7d4f98b5fd-ltzh8 Add eth0 [10.134.0.127/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-mt-01955278-7d4f98b5fd-ltzh8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-01955278-7d4f98b5fd Created pod: payload-processing-e2e-mt-01955278-7d4f98b5fd-ltzh8 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-01955278-f55689567 Error creating: pods "payload-pre-processing-e2e-mt-01955278-f55689567-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-01955278: serviceaccount "payload-processing-e2e-mt-01955278" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-01955278-f55689567 Created pod: payload-pre-processing-e2e-mt-01955278-f55689567-97lbq 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-01955278 Scaled up replica set payload-pre-processing-e2e-mt-01955278-f55689567 from 0 to 1 55m Warning FailedCreate replicaset/payload-processing-e2e-mt-01955278-7d4f98b5fd Error creating: pods "payload-processing-e2e-mt-01955278-7d4f98b5fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-01955278: serviceaccount "payload-processing-e2e-mt-01955278" not found 55m Normal Created pod/payload-processing-e2e-mt-01955278-7d4f98b5fd-ltzh8 Container created 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-0efe43-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal Pulled pod/payload-pre-processing-e2e-mt-01955278-f55689567-97lbq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-pre-processing-e2e-mt-01955278-f55689567-97lbq Container created 55m Normal Started pod/payload-pre-processing-e2e-mt-01955278-f55689567-97lbq Container started 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-0efe43-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-e371c3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-e371c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled down replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 10 to 6 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Readiness probe failed: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 54m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-mqc7s Readiness probe error: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md 54m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-xlp9c Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-tr5m4 Stopping container istio-proxy 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 6; reason: All metrics below target 54m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-qtq79 Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7r7md Stopping container istio-proxy 54m Normal AddedLabel gateway/e2e-shared-b-w3-228454 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-228454 54m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-228454-openshift-default Scaled up replica set e2e-shared-b-w3-228454-openshift-default-6476dbf645 from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-228454-openshift-default-6476dbf645 Created pod: e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b 54m Normal AddedInterface pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b Add eth0 [10.132.0.92/23] from ovn-kubernetes 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-9dba4af9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-mt-9dba4af9-openshift-default-7d7d889b5 Created pod: e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 54m Normal ScalingReplicaSet deployment/e2e-mt-9dba4af9-openshift-default Scaled up replica set e2e-mt-9dba4af9-openshift-default-7d7d889b5 from 1 to 2 54m Normal Pulled pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 Add eth0 [10.133.0.169/23] from ovn-kubernetes 54m Normal Created pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 Container created 54m Normal Started pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 Container started 54m Normal Pulled pod/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq Container started 54m Normal Created pod/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq Container created 54m Warning ProbeError pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 Startup probe error: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 Startup probe failed: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused 54m Warning ProbeError pod/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq Startup probe error: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq Startup probe failed: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-228454 Scaled up replica set payload-processing-e2e-shared-b-w3-228454-df8d85c7d from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-228454-df8d85c7d Error creating: pods "payload-processing-e2e-shared-b-w3-228454-df8d85c7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-228454: serviceaccount "payload-processing-e2e-shared-b-w3-228454" not found 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-228454 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47 from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-228454-df8d85c7d Created pod: payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-228454: serviceaccount "payload-processing-e2e-shared-b-w3-228454" not found 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47 Created pod: payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 54m Normal Created pod/payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw Container created 54m Normal Pulled pod/payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 Container started 54m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 Container created 54m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw Add eth0 [10.134.0.130/23] from ovn-kubernetes 54m Normal Started pod/payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 Add eth0 [10.134.0.131/23] from ovn-kubernetes 54m Normal Started pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Container started 54m Normal Pulled pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Container created 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-a59ecb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-a59ecb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv 54m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-r4qqv Stopping container istio-proxy 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 5; reason: All metrics below target 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-6ck5k 54m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-2c9jd 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled down replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 10 to 5 54m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-nbnsl Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Stopping container istio-proxy 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-5704d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Readiness probe failed: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Readiness probe error: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-5704d6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning ProbeError pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Startup probe error: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Startup probe failed: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Readiness probe error: HTTP probe failed with statuscode: 503... 54m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-xnfpw Readiness probe failed: HTTP probe failed with statuscode: 503 54m Normal AddedLabel gateway/e2e-api-a-972fee Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-972fee 54m Normal ScalingReplicaSet deployment/e2e-api-a-972fee-openshift-default Scaled up replica set e2e-api-a-972fee-openshift-default-85db59bcbf from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-api-a-972fee-openshift-default-85db59bcbf Created pod: e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p 54m Normal AddedInterface pod/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p Add eth0 [10.133.0.171/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-api-a-d727dc-openshift-default-d7cb66b4c Created pod: e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 54m Normal AddedLabel gateway/e2e-api-a-d727dc Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d727dc 54m Normal ScalingReplicaSet deployment/e2e-api-a-d727dc-openshift-default Scaled up replica set e2e-api-a-d727dc-openshift-default-d7cb66b4c from 0 to 1 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-01955278-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Normal AddedInterface pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Add eth0 [10.133.0.172/23] from ovn-kubernetes 54m Normal Started pod/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc Container started 54m Normal Created pod/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc Container created 54m Normal Pulled pod/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-01955278-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-5704d6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-5704d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-d727dc-db644f5ff Created pod: payload-pre-processing-e2e-api-a-d727dc-db644f5ff-cg8nn 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-d727dc-bc4fc7c88 Created pod: payload-processing-e2e-api-a-d727dc-bc4fc7c88-6sw2f 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-d727dc Scaled up replica set payload-pre-processing-e2e-api-a-d727dc-db644f5ff from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-api-a-d727dc-bc4fc7c88 Error creating: pods "payload-processing-e2e-api-a-d727dc-bc4fc7c88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d727dc: serviceaccount "payload-processing-e2e-api-a-d727dc" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-d727dc Scaled up replica set payload-processing-e2e-api-a-d727dc-bc4fc7c88 from 0 to 1 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-d727dc-db644f5ff Error creating: pods "payload-pre-processing-e2e-api-a-d727dc-db644f5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d727dc: serviceaccount "payload-processing-e2e-api-a-d727dc" not found 53m Normal Created pod/payload-processing-e2e-api-a-d727dc-bc4fc7c88-6sw2f Container created 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-d727dc-db644f5ff-cg8nn Add eth0 [10.134.0.133/23] from ovn-kubernetes 53m Warning Unhealthy pod/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc Startup probe failed: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused 53m Normal Started pod/payload-processing-e2e-api-a-d727dc-bc4fc7c88-6sw2f Container started 53m Warning ProbeError pod/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc Startup probe error: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused... 53m Normal Pulled pod/payload-pre-processing-e2e-api-a-d727dc-db644f5ff-cg8nn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Pulled pod/payload-processing-e2e-api-a-d727dc-bc4fc7c88-6sw2f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-api-a-d727dc-db644f5ff-cg8nn Container started 53m Normal Created pod/payload-pre-processing-e2e-api-a-d727dc-db644f5ff-cg8nn Container created 53m Normal AddedInterface pod/payload-processing-e2e-api-a-d727dc-bc4fc7c88-6sw2f Add eth0 [10.134.0.134/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-972fee Scaled up replica set payload-processing-e2e-api-a-972fee-546d4b6686 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-972fee-546d4b6686 Created pod: payload-processing-e2e-api-a-972fee-546d4b6686-svw78 53m Warning FailedCreate replicaset/payload-processing-e2e-api-a-972fee-546d4b6686 Error creating: pods "payload-processing-e2e-api-a-972fee-546d4b6686-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-972fee: serviceaccount "payload-processing-e2e-api-a-972fee" not found 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-972fee-fdbb7989f Error creating: pods "payload-pre-processing-e2e-api-a-972fee-fdbb7989f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-972fee: serviceaccount "payload-processing-e2e-api-a-972fee" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-972fee Scaled up replica set payload-pre-processing-e2e-api-a-972fee-fdbb7989f from 0 to 1 53m Normal AddedInterface pod/payload-processing-e2e-api-a-972fee-546d4b6686-svw78 Add eth0 [10.134.0.135/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-972fee-fdbb7989f Created pod: payload-pre-processing-e2e-api-a-972fee-fdbb7989f-mdfj4 53m Normal Created pod/payload-processing-e2e-api-a-972fee-546d4b6686-svw78 Container created 53m Normal Created pod/payload-pre-processing-e2e-api-a-972fee-fdbb7989f-mdfj4 Container created 53m Normal Started pod/payload-pre-processing-e2e-api-a-972fee-fdbb7989f-mdfj4 Container started 53m Normal Pulled pod/payload-processing-e2e-api-a-972fee-546d4b6686-svw78 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-972fee-fdbb7989f-mdfj4 Add eth0 [10.132.0.93/23] from ovn-kubernetes 53m Normal Started pod/payload-processing-e2e-api-a-972fee-546d4b6686-svw78 Container started 53m Normal Pulled pod/payload-pre-processing-e2e-api-a-972fee-fdbb7989f-mdfj4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-a59ecb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-a59ecb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Killing pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr Stopping container payload-processing 53m Normal Killing pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl Stopping container payload-pre-processing 53m Normal SuccessfulCreate replicaset/e2e-api-a-d74808-openshift-default-5454f78567 Created pod: e2e-api-a-d74808-openshift-default-5454f78567-g69q7 53m Normal ScalingReplicaSet deployment/e2e-api-a-d74808-openshift-default Scaled up replica set e2e-api-a-d74808-openshift-default-5454f78567 from 0 to 1 53m Normal AddedLabel gateway/e2e-api-a-d74808 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d74808 53m Normal AddedInterface pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Add eth0 [10.133.0.174/23] from ovn-kubernetes 53m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 Stopping container istio-proxy 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-01955278-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-lhgm4 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled down replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 10 to 9 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 9; reason: All metrics below target 53m Normal Pulled pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-01955278-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Created pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b Container created 53m Normal Started pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b Container started 53m Warning Unhealthy pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-6kdzl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-d74808-645df45b6c Error creating: pods "payload-pre-processing-e2e-api-a-d74808-645df45b6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d74808: serviceaccount "payload-processing-e2e-api-a-d74808" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-d74808 Scaled up replica set payload-processing-e2e-api-a-d74808-5ff59bf748 from 0 to 1 53m Warning Unhealthy pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-4fmbr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-d74808 Scaled up replica set payload-pre-processing-e2e-api-a-d74808-645df45b6c from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-api-a-d74808-5ff59bf748 Error creating: pods "payload-processing-e2e-api-a-d74808-5ff59bf748-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d74808: serviceaccount "payload-processing-e2e-api-a-d74808" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-d74808-5ff59bf748 Created pod: payload-processing-e2e-api-a-d74808-5ff59bf748-5cpkf 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-d74808-645df45b6c Created pod: payload-pre-processing-e2e-api-a-d74808-645df45b6c-h4f6c 53m Normal Pulled pod/payload-processing-e2e-api-a-d74808-5ff59bf748-5cpkf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-api-a-d74808-5ff59bf748-5cpkf Container created 53m Normal Started pod/payload-processing-e2e-api-a-d74808-5ff59bf748-5cpkf Container started 53m Normal AddedInterface pod/payload-processing-e2e-api-a-d74808-5ff59bf748-5cpkf Add eth0 [10.134.0.137/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-d74808-645df45b6c-h4f6c Add eth0 [10.134.0.136/23] from ovn-kubernetes 53m Normal Started pod/payload-pre-processing-e2e-api-a-d74808-645df45b6c-h4f6c Container started 53m Normal Pulled pod/payload-pre-processing-e2e-api-a-d74808-645df45b6c-h4f6c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-api-a-d74808-645df45b6c-h4f6c Container created 53m Warning ProbeError pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 53m Normal ScalingReplicaSet deployment/e2e-api-a-2b206b-openshift-default Scaled up replica set e2e-api-a-2b206b-openshift-default-6d945cc57c from 0 to 1 53m Normal AddedLabel gateway/e2e-api-a-2b206b Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-2b206b 53m Normal SuccessfulCreate replicaset/e2e-api-a-2b206b-openshift-default-6d945cc57c Created pod: e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v 53m Normal AddedInterface pod/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v Add eth0 [10.133.0.176/23] from ovn-kubernetes 53m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk Stopping container istio-proxy 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 6; reason: All metrics below target 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj 53m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-2pvwv Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-mfhnr 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled down replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 10 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 5; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled down replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 9 to 6 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp 53m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-tg5tj Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9c658 Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-9pkxs Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zqcnk 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-w8th6 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d74808-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d74808-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-2b206b-55f8fbf4 Created pod: payload-processing-e2e-api-a-2b206b-55f8fbf4-lpgqq 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79 Error creating: pods "payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2b206b: serviceaccount "payload-processing-e2e-api-a-2b206b" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79 Created pod: payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-5s7b7 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-2b206b Scaled up replica set payload-pre-processing-e2e-api-a-2b206b-5745bc7d79 from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-2b206b Scaled up replica set payload-processing-e2e-api-a-2b206b-55f8fbf4 from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-api-a-2b206b-55f8fbf4 Error creating: pods "payload-processing-e2e-api-a-2b206b-55f8fbf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2b206b: serviceaccount "payload-processing-e2e-api-a-2b206b" not found 53m Normal Pulled pod/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-5s7b7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-5s7b7 Add eth0 [10.132.0.95/23] from ovn-kubernetes 53m Normal Created pod/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-5s7b7 Container created 53m Normal AddedInterface pod/payload-processing-e2e-api-a-2b206b-55f8fbf4-lpgqq Add eth0 [10.134.0.138/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-api-a-2b206b-55f8fbf4-lpgqq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-5s7b7 Container started 53m Normal Started pod/payload-processing-e2e-api-a-2b206b-55f8fbf4-lpgqq Container started 53m Normal Created pod/payload-processing-e2e-api-a-2b206b-55f8fbf4-lpgqq Container created 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-228454-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-228454-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Readiness probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-cmxbp Readiness probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-2b206b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-2b206b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 8; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled down replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 10 to 8 53m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-t5cm9 53m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-4vjwq 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h 53m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled down replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 10 to 6 53m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb Stopping container istio-proxy 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 6; reason: All metrics below target 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-ns4nb 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-vnlbf 53m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wr62h Stopping container istio-proxy 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-228454-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-228454-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-vkszr Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 52m Normal Created pod/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p Container created 52m Normal Started pod/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p Container started 52m Normal Pulled pod/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Container created 52m Normal Pulled pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Container started 52m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf Stopping container istio-proxy 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 6; reason: All metrics below target 52m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-28ksf 52m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb 52m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-gphbb Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default Scaled down replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 8 to 6 52m Normal ScalingReplicaSet deployment/e2e-api-b-b5ea88-openshift-default Scaled up replica set e2e-api-b-b5ea88-openshift-default-845d49979c from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-api-b-b5ea88-openshift-default-845d49979c Created pod: e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc 52m Warning ProbeError pod/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p Startup probe error: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused... 52m Normal AddedLabel gateway/e2e-api-b-b5ea88 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-b5ea88 52m Warning Unhealthy pod/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p Startup probe failed: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused... 52m Normal AddedInterface pod/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc Add eth0 [10.133.0.179/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused 52m Warning FailedCreate replicaset/payload-processing-e2e-api-b-b5ea88-59d675d48b Error creating: pods "payload-processing-e2e-api-b-b5ea88-59d675d48b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-b5ea88: serviceaccount "payload-processing-e2e-api-b-b5ea88" not found 52m Normal SuccessfulCreate replicaset/e2e-api-b-aab379-openshift-default-69594d6c89 Created pod: e2e-api-b-aab379-openshift-default-69594d6c89-wpdld 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-b5ea88 Scaled up replica set payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9 from 0 to 1 52m Normal AddedLabel gateway/e2e-api-b-aab379 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-aab379 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9 Error creating: pods "payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-b5ea88: serviceaccount "payload-processing-e2e-api-b-b5ea88" not found 52m Normal ScalingReplicaSet deployment/e2e-api-b-aab379-openshift-default Scaled up replica set e2e-api-b-aab379-openshift-default-69594d6c89 from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-b5ea88 Scaled up replica set payload-processing-e2e-api-b-b5ea88-59d675d48b from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9 Created pod: payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-5cv2v 52m Normal AddedInterface pod/payload-processing-e2e-api-b-b5ea88-59d675d48b-pwrj9 Add eth0 [10.133.0.182/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Add eth0 [10.133.0.181/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-b5ea88-59d675d48b Created pod: payload-processing-e2e-api-b-b5ea88-59d675d48b-pwrj9 52m Normal Started pod/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-5cv2v Container started 52m Normal Started pod/payload-processing-e2e-api-b-b5ea88-59d675d48b-pwrj9 Container started 52m Normal Created pod/payload-processing-e2e-api-b-b5ea88-59d675d48b-pwrj9 Container created 52m Normal Pulled pod/payload-processing-e2e-api-b-b5ea88-59d675d48b-pwrj9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Pulled pod/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-5cv2v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-5cv2v Add eth0 [10.133.0.183/23] from ovn-kubernetes 52m Normal Created pod/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-5cv2v Container created 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-aab379-54cb856f5 Created pod: payload-pre-processing-e2e-api-b-aab379-54cb856f5-6bvr9 52m Warning FailedCreate replicaset/payload-processing-e2e-api-b-aab379-5c6c8cf798 Error creating: pods "payload-processing-e2e-api-b-aab379-5c6c8cf798-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-aab379: serviceaccount "payload-processing-e2e-api-b-aab379" not found 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-aab379-54cb856f5 Error creating: pods "payload-pre-processing-e2e-api-b-aab379-54cb856f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-aab379: serviceaccount "payload-processing-e2e-api-b-aab379" not found 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-aab379 Scaled up replica set payload-processing-e2e-api-b-aab379-5c6c8cf798 from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-aab379 Scaled up replica set payload-pre-processing-e2e-api-b-aab379-54cb856f5 from 0 to 1 52m Normal Started pod/payload-pre-processing-e2e-api-b-aab379-54cb856f5-6bvr9 Container started 52m Normal Created pod/payload-pre-processing-e2e-api-b-aab379-54cb856f5-6bvr9 Container created 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-aab379-5c6c8cf798 Created pod: payload-processing-e2e-api-b-aab379-5c6c8cf798-x5xnl 52m Normal Pulled pod/payload-pre-processing-e2e-api-b-aab379-54cb856f5-6bvr9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-aab379-54cb856f5-6bvr9 Add eth0 [10.133.0.184/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-api-b-aab379-5c6c8cf798-x5xnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-api-b-aab379-5c6c8cf798-x5xnl Container created 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d727dc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d727dc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal AddedInterface pod/payload-processing-e2e-api-b-aab379-5c6c8cf798-x5xnl Add eth0 [10.133.0.185/23] from ovn-kubernetes 52m Normal Started pod/payload-processing-e2e-api-b-aab379-5c6c8cf798-x5xnl Container started 52m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs 52m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default Scaled down replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 6 to 5 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 5; reason: All metrics below target 52m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-f6tdj 52m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-q5vhs Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default Scaled down replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 6 to 5 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 5; reason: All metrics below target 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-9dba4af9 Scaled up replica set payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47 from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-9dba4af9 Scaled up replica set payload-processing-e2e-mt-9dba4af9-6d98fd4964 from 0 to 1 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled down replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 6 to 5 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 5; reason: All metrics below target 52m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc 52m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-d67bc Stopping container istio-proxy 52m Warning FailedCreate replicaset/payload-processing-e2e-mt-9dba4af9-6d98fd4964 Error creating: pods "payload-processing-e2e-mt-9dba4af9-6d98fd4964-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9dba4af9: serviceaccount "payload-processing-e2e-mt-9dba4af9" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-9dba4af9-6d98fd4964 Created pod: payload-processing-e2e-mt-9dba4af9-6d98fd4964-x5f5d 52m Normal AddedInterface pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-x5f5d Add eth0 [10.133.0.186/23] from ovn-kubernetes 52m Normal AddedInterface pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-h5tsw Add eth0 [10.133.0.187/23] from ovn-kubernetes 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47 Error creating: pods "payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9dba4af9: serviceaccount "payload-processing-e2e-mt-9dba4af9" not found 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47 Created pod: payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-h5tsw 52m Normal Pulled pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-x5f5d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-x5f5d Container created 52m Normal Pulled pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-h5tsw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-h5tsw Container created 52m Normal Started pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-h5tsw Container started 52m Normal Started pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-x5f5d Container started 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-972fee-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-972fee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal Killing pod/payload-processing-e2e-mt-01955278-7d4f98b5fd-ltzh8 Stopping container payload-processing 52m Normal Killing pod/payload-pre-processing-e2e-mt-01955278-f55689567-97lbq Stopping container payload-pre-processing 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d727dc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d727dc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-b5ea88-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-b5ea88-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal Created pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Container created 52m Normal Started pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Container started 52m Normal Pulled pod/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning ProbeError pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Readiness probe error: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Readiness probe failed: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused 52m Normal Started pod/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v Container started 52m Normal Created pod/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v Container created 52m Normal Pulled pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning Unhealthy pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Startup probe failed: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v Startup probe error: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Startup probe error: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v Startup probe failed: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused 52m Normal AddedLabel gateway/e2e-api-b-2a2835 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-2a2835 52m Normal SuccessfulCreate replicaset/e2e-api-b-2a2835-openshift-default-74dc4b7c9c Created pod: e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr 52m Normal ScalingReplicaSet deployment/e2e-api-b-2a2835-openshift-default Scaled up replica set e2e-api-b-2a2835-openshift-default-74dc4b7c9c from 0 to 1 52m Normal AddedInterface pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Add eth0 [10.133.0.189/23] from ovn-kubernetes 52m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 52m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc Stopping container istio-proxy 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default New size: 4; reason: All metrics below target 52m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t Stopping container istio-proxy 52m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-wsnc4 Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled down replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 7 to 4 52m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-cftgc 52m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-jch2t 51m Normal ScalingReplicaSet deployment/e2e-api-b-1c4b82-openshift-default Scaled up replica set e2e-api-b-1c4b82-openshift-default-7f9f99f747 from 0 to 1 51m Normal AddedLabel gateway/e2e-api-b-1c4b82 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-1c4b82 51m Normal SuccessfulCreate replicaset/e2e-api-b-1c4b82-openshift-default-7f9f99f747 Created pod: e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt 51m Normal AddedInterface pod/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt Add eth0 [10.133.0.191/23] from ovn-kubernetes 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-2b206b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-2b206b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedCreate replicaset/payload-processing-e2e-api-b-1c4b82-55c456d68d Error creating: pods "payload-processing-e2e-api-b-1c4b82-55c456d68d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-1c4b82: serviceaccount "payload-processing-e2e-api-b-1c4b82" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-1c4b82 Scaled up replica set payload-processing-e2e-api-b-1c4b82-55c456d68d from 0 to 1 51m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-pkl7b Add eth0 [10.134.0.140/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-1c4b82 Scaled up replica set payload-pre-processing-e2e-api-b-1c4b82-547d7565c9 from 0 to 1 51m Normal Pulled pod/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-pkl7b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-1c4b82-55c456d68d Created pod: payload-processing-e2e-api-b-1c4b82-55c456d68d-4gnsj 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9 Created pod: payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-pkl7b 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9 Error creating: pods "payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-1c4b82: serviceaccount "payload-processing-e2e-api-b-1c4b82" not found 51m Normal Created pod/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-pkl7b Container created 51m Normal Started pod/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-pkl7b Container started 51m Normal Started pod/payload-processing-e2e-api-b-1c4b82-55c456d68d-4gnsj Container started 51m Normal Created pod/payload-processing-e2e-api-b-1c4b82-55c456d68d-4gnsj Container created 51m Normal Pulled pod/payload-processing-e2e-api-b-1c4b82-55c456d68d-4gnsj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-api-b-1c4b82-55c456d68d-4gnsj Add eth0 [10.134.0.141/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5 Created pod: payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-m54k4 51m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-m54k4 Add eth0 [10.134.0.143/23] from ovn-kubernetes 51m Normal AddedInterface pod/payload-processing-e2e-api-b-2a2835-c75b7cc9d-shj8b Add eth0 [10.134.0.142/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-2a2835-c75b7cc9d Created pod: payload-processing-e2e-api-b-2a2835-c75b7cc9d-shj8b 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-2a2835 Scaled up replica set payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5 from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-2a2835 Scaled up replica set payload-processing-e2e-api-b-2a2835-c75b7cc9d from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5 Error creating: pods "payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-2a2835: serviceaccount "payload-processing-e2e-api-b-2a2835" not found 51m Warning FailedCreate replicaset/payload-processing-e2e-api-b-2a2835-c75b7cc9d Error creating: pods "payload-processing-e2e-api-b-2a2835-c75b7cc9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-2a2835: serviceaccount "payload-processing-e2e-api-b-2a2835" not found 51m Normal Pulled pod/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-m54k4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-m54k4 Container started 51m Normal Created pod/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-m54k4 Container created 51m Normal Pulled pod/payload-processing-e2e-api-b-2a2835-c75b7cc9d-shj8b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-processing-e2e-api-b-2a2835-c75b7cc9d-shj8b Container created 51m Normal Started pod/payload-processing-e2e-api-b-2a2835-c75b7cc9d-shj8b Container started 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d74808-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d74808-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Killing pod/e2e-mt-01955278-openshift-default-56c64d6797-9hfnc Stopping container istio-proxy 51m Normal AddedLabel gateway/e2e-inf-a-5a9d7b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-5a9d7b 51m Normal ScalingReplicaSet deployment/e2e-inf-a-5a9d7b-openshift-default Scaled up replica set e2e-inf-a-5a9d7b-openshift-default-796bc9cccf from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf Created pod: e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t 51m Normal SuccessfulCreate replicaset/e2e-api-a-4bf33a-openshift-default-b849dbcc5 Created pod: e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 51m Normal ScalingReplicaSet deployment/e2e-api-a-4bf33a-openshift-default Scaled up replica set e2e-api-a-4bf33a-openshift-default-b849dbcc5 from 0 to 1 51m Normal AddedLabel gateway/e2e-api-a-4bf33a Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-4bf33a 51m Normal AddedInterface pod/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t Add eth0 [10.133.0.193/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Add eth0 [10.133.0.194/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-5a9d7b Scaled up replica set payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8 Created pod: payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9 Error creating: pods "payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-5a9d7b: serviceaccount "payload-processing-e2e-inf-a-5a9d7b" not found 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9 Created pod: payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-5a9d7b Scaled up replica set payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8 from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8 Error creating: pods "payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-5a9d7b: serviceaccount "payload-processing-e2e-inf-a-5a9d7b" not found 51m Normal Created pod/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh Container created 51m Normal Created pod/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx Container created 51m Normal Pulled pod/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh Add eth0 [10.132.0.97/23] from ovn-kubernetes 51m Normal Started pod/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx Container started 51m Normal AddedInterface pod/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx Add eth0 [10.134.0.144/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh Container started 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-4bf33a Scaled up replica set payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55 from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-api-a-4bf33a-6bb59459b Error creating: pods "payload-processing-e2e-api-a-4bf33a-6bb59459b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-4bf33a: serviceaccount "payload-processing-e2e-api-a-4bf33a" not found 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55 Error creating: pods "payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-4bf33a: serviceaccount "payload-processing-e2e-api-a-4bf33a" not found 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55 Created pod: payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-9mjfq 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-4bf33a Scaled up replica set payload-processing-e2e-api-a-4bf33a-6bb59459b from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-4bf33a-6bb59459b Created pod: payload-processing-e2e-api-a-4bf33a-6bb59459b-4qdp6 51m Normal Pulled pod/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-9mjfq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Pulled pod/payload-processing-e2e-api-a-4bf33a-6bb59459b-4qdp6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-9mjfq Container created 51m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-9mjfq Add eth0 [10.132.0.99/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-9mjfq Container started 51m Normal Started pod/payload-processing-e2e-api-a-4bf33a-6bb59459b-4qdp6 Container started 51m Normal AddedInterface pod/payload-processing-e2e-api-a-4bf33a-6bb59459b-4qdp6 Add eth0 [10.132.0.98/23] from ovn-kubernetes 51m Normal Created pod/payload-processing-e2e-api-a-4bf33a-6bb59459b-4qdp6 Container created 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default New size: 5; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-7zkr8 51m Normal Started pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Container started 51m Normal Created pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Container created 51m Normal Pulled pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-aab379-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-aab379-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning Unhealthy pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 51m Warning ProbeError pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 51m Normal Killing pod/payload-processing-e2e-mt-9dba4af9-6d98fd4964-x5f5d Stopping container payload-processing 51m Normal Killing pod/payload-pre-processing-e2e-mt-9dba4af9-7f54cffb47-h5tsw Stopping container payload-pre-processing 51m Normal Started pod/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc Container started 51m Normal Created pod/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc Container created 51m Normal Pulled pod/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Killing pod/payload-processing-e2e-api-a-d727dc-bc4fc7c88-6sw2f Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-api-a-d727dc-db644f5ff-cg8nn Stopping container payload-pre-processing 50m Warning Unhealthy pod/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused 50m Warning ProbeError pod/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused... 50m Normal Killing pod/payload-processing-e2e-api-a-972fee-546d4b6686-svw78 Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-api-a-972fee-fdbb7989f-mdfj4 Stopping container payload-pre-processing 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-aab379-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-aab379-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Pulled pod/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt Container created 50m Normal Started pod/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt Container started 50m Warning ProbeError pod/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt Startup probe error: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt Startup probe failed: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-b5ea88-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-b5ea88-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default New size: 4; reason: All metrics below target 50m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 50m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-sx4vw Stopping container istio-proxy 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default New size: 4; reason: All metrics below target 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk 50m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 4; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls Stopping container istio-proxy 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 4; reason: All metrics below target 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled down replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 5 to 4 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 4; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-qpdxk Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-pndg4 Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-lw5ls 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled down replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 5 to 4 50m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-7cjsk Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled down replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 5 to 4 50m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-dgbv6 Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning ProbeError pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Readiness probe error: Get "http://10.133.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Readiness probe failed: Get "http://10.133.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Normal Created pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Container created 50m Normal Started pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Container started 50m Normal Started pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Container started 50m Normal AddedInterface pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Add eth0 [10.132.0.106/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/e2e-api-b-2a2835-openshift-default Scaled up replica set e2e-api-b-2a2835-openshift-default-74dc4b7c9c from 1 to 4 50m Normal Pulled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Add eth0 [10.134.0.145/23] from ovn-kubernetes 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-2a2835-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal Killing pod/payload-processing-e2e-api-a-d74808-5ff59bf748-5cpkf Stopping container payload-processing 50m Normal SuccessfulCreate replicaset/e2e-api-b-2a2835-openshift-default-74dc4b7c9c Created pod: e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 50m Normal SuccessfulCreate replicaset/e2e-api-b-2a2835-openshift-default-74dc4b7c9c Created pod: e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z 50m Normal SuccessfulCreate replicaset/e2e-api-b-2a2835-openshift-default-74dc4b7c9c Created pod: e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 50m Normal Started pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Container started 50m Normal Created pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Container created 50m Normal Killing pod/payload-pre-processing-e2e-api-a-d74808-645df45b6c-h4f6c Stopping container payload-pre-processing 50m Normal Pulled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Add eth0 [10.132.0.105/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Container created 50m Normal Created pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Container created 50m Normal Started pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Container started 50m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s 50m Normal Pulled pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Container started 50m Normal Killing pod/payload-processing-e2e-api-a-2b206b-55f8fbf4-lpgqq Stopping container payload-processing 50m Normal Created pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Container created 50m Normal Killing pod/payload-pre-processing-e2e-api-a-2b206b-5745bc7d79-5s7b7 Stopping container payload-pre-processing 50m Normal Started pod/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t Container started 50m Normal Created pod/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t Container created 50m Normal Pulled pod/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Stopping container istio-proxy 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-5a9d7b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-p7d9s Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-5a9d7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Startup probe error: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Startup probe failed: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Startup probe error: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t Startup probe error: Get "http://10.133.0.193:15021/healthz/ready": dial tcp 10.133.0.193:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t Startup probe failed: Get "http://10.133.0.193:15021/healthz/ready": dial tcp 10.133.0.193:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Startup probe error: Get "http://10.133.0.189:15021/healthz/ready": dial tcp 10.133.0.189:15021: connect: connection refused... 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-1c4b82-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Startup probe failed: Get "http://10.133.0.189:15021/healthz/ready": dial tcp 10.133.0.189:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Startup probe failed: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-1c4b82-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal SuccessfulCreate replicaset/e2e-inf-b-ca02c9-openshift-default-67db76697 Created pod: e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv 49m Normal AddedLabel gateway/e2e-inf-b-ca02c9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ca02c9 49m Normal ScalingReplicaSet deployment/e2e-inf-b-ca02c9-openshift-default Scaled up replica set e2e-inf-b-ca02c9-openshift-default-67db76697 from 0 to 1 49m Normal AddedInterface pod/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv Add eth0 [10.134.0.146/23] from ovn-kubernetes 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-2a2835-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-2a2835-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning Unhealthy pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning ProbeError pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8 Error creating: pods "payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ca02c9: serviceaccount "payload-processing-e2e-inf-b-ca02c9" not found 49m Normal Killing pod/e2e-api-a-d727dc-openshift-default-d7cb66b4c-dvll9 Stopping container istio-proxy 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-ca02c9 Scaled up replica set payload-processing-e2e-inf-b-ca02c9-7fc799d5c8 from 0 to 1 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-ca02c9 Scaled up replica set payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff Created pod: payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8 Created pod: payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-pkcxp 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff Error creating: pods "payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ca02c9: serviceaccount "payload-processing-e2e-inf-b-ca02c9" not found 49m Normal Created pod/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h Container created 49m Normal Started pod/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h Container started 49m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h Add eth0 [10.134.0.147/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-pkcxp Container started 49m Normal Created pod/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-pkcxp Container created 49m Normal Pulled pod/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-pkcxp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-pkcxp Add eth0 [10.132.0.107/23] from ovn-kubernetes 49m Normal Killing pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-6f7w7 Stopping container istio-proxy 49m Normal Killing pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Stopping container istio-proxy 49m Warning Unhealthy pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning ProbeError pod/e2e-mt-9dba4af9-openshift-default-7d7d889b5-kgllg Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-4bf33a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-4bf33a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal AddedLabel gateway/e2e-api-b-ea9ed2 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-ea9ed2 49m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg 49m Normal ScalingReplicaSet deployment/e2e-api-b-ea9ed2-openshift-default Scaled up replica set e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 from 0 to 1 49m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Add eth0 [10.133.0.200/23] from ovn-kubernetes 49m Normal Killing pod/e2e-api-a-972fee-openshift-default-85db59bcbf-bs28p Stopping container istio-proxy 49m Normal Killing pod/payload-pre-processing-e2e-api-b-b5ea88-55cc6694f9-5cv2v Stopping container payload-pre-processing 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-2a2835-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-2a2835-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Killing pod/payload-processing-e2e-api-b-b5ea88-59d675d48b-pwrj9 Stopping container payload-processing 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc Error creating: pods "payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-ea9ed2: serviceaccount "payload-processing-e2e-api-b-ea9ed2" not found 49m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Stopping container istio-proxy 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-ea9ed2 Scaled up replica set payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb Created pod: payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-ftnvk 49m Warning FailedCreate replicaset/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb Error creating: pods "payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-ea9ed2: serviceaccount "payload-processing-e2e-api-b-ea9ed2" not found 49m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 3; reason: All metrics below target 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled down replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 4 to 3 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc Created pod: payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-vp88x 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-ea9ed2 Scaled up replica set payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc from 0 to 1 49m Normal Pulled pod/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-vp88x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-ftnvk Add eth0 [10.132.0.112/23] from ovn-kubernetes 49m Normal Started pod/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-ftnvk Container started 49m Normal Created pod/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-ftnvk Container created 49m Normal Pulled pod/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-ftnvk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-vp88x Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-vp88x Add eth0 [10.132.0.111/23] from ovn-kubernetes 49m Normal Started pod/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-vp88x Container started 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-5a9d7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-5a9d7b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-mv7fc Readiness probe failed: HTTP probe failed with statuscode: 503 49m Normal Killing pod/payload-processing-e2e-api-b-aab379-5c6c8cf798-x5xnl Stopping container payload-processing 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-4bf33a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-4bf33a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Killing pod/payload-pre-processing-e2e-api-b-aab379-54cb856f5-6bvr9 Stopping container payload-pre-processing 49m Normal ScalingReplicaSet deployment/e2e-inf-a-648d5f-openshift-default Scaled up replica set e2e-inf-a-648d5f-openshift-default-86d675f7b9 from 0 to 1 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-ea9ed2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal AddedInterface pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Add eth0 [10.133.0.201/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-inf-a-648d5f-openshift-default-86d675f7b9 Created pod: e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh 49m Normal AddedLabel gateway/e2e-inf-a-648d5f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-648d5f 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-ea9ed2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal SuccessfulCreate replicaset/e2e-api-b-2a2835-openshift-default-74dc4b7c9c Created pod: e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd 49m Normal ScalingReplicaSet deployment/e2e-api-b-2a2835-openshift-default Scaled up replica set e2e-api-b-2a2835-openshift-default-74dc4b7c9c from 4 to 6 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-2a2835-openshift-default New size: 6; reason: All metrics below target 49m Normal AddedInterface pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Add eth0 [10.133.0.202/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-api-b-2a2835-openshift-default-74dc4b7c9c Created pod: e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc 49m Normal AddedInterface pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Add eth0 [10.134.0.148/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-648d5f-54b7c56458 Error creating: pods "payload-processing-e2e-inf-a-648d5f-54b7c56458-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-648d5f: serviceaccount "payload-processing-e2e-inf-a-648d5f" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-648d5f-54b7c56458 Created pod: payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-648d5f Scaled up replica set payload-processing-e2e-inf-a-648d5f-54b7c56458 from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59 Error creating: pods "payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-648d5f: serviceaccount "payload-processing-e2e-inf-a-648d5f" not found 49m Normal Created pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Container created 49m Normal Started pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Container started 49m Normal Created pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Container created 49m Normal Pulled pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Container started 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-648d5f Scaled up replica set payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59 Created pod: payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc 49m Normal Pulled pod/payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 Container started 49m Normal Pulled pod/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc Add eth0 [10.133.0.203/23] from ovn-kubernetes 49m Normal AddedInterface pod/payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 Add eth0 [10.132.0.113/23] from ovn-kubernetes 49m Normal Started pod/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc Container started 49m Normal Created pod/payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 Container created 49m Normal Created pod/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc Container created 49m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Startup probe error: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Startup probe failed: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused 49m Normal Killing pod/e2e-api-a-d74808-openshift-default-5454f78567-g69q7 Stopping container istio-proxy 49m Normal Killing pod/e2e-api-a-2b206b-openshift-default-6d945cc57c-9wr8v Stopping container istio-proxy 48m Normal Killing pod/payload-processing-e2e-api-b-2a2835-c75b7cc9d-shj8b Stopping container payload-processing 48m Normal Killing pod/payload-pre-processing-e2e-api-b-2a2835-5b9b7c4cf5-m54k4 Stopping container payload-pre-processing 48m Normal Killing pod/payload-pre-processing-e2e-api-b-1c4b82-547d7565c9-pkl7b Stopping container payload-pre-processing 48m Normal Killing pod/payload-processing-e2e-api-b-1c4b82-55c456d68d-4gnsj Stopping container payload-processing 48m Normal Pulled pod/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv Container created 48m Normal Started pod/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv Container started 48m Warning ProbeError pod/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 48m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z 48m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-s5k6z Stopping container istio-proxy 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ca02c9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ca02c9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled down replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 4 to 3 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 3; reason: All metrics below target 48m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc 48m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-5hnmc Stopping container istio-proxy 48m Normal Killing pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-inf-a-d15602-openshift-default Scaled up replica set e2e-inf-a-d15602-openshift-default-6b94954795 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-a-d15602-openshift-default-6b94954795 Created pod: e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d 48m Normal AddedLabel gateway/e2e-inf-a-d15602 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-d15602 48m Warning Unhealthy pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning ProbeError pod/e2e-api-b-aab379-openshift-default-69594d6c89-wpdld Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-ea9ed2-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 48m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt 48m Normal ScalingReplicaSet deployment/e2e-api-b-ea9ed2-openshift-default Scaled up replica set e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 from 1 to 4 48m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 Add eth0 [10.132.0.116/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Add eth0 [10.133.0.207/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt Add eth0 [10.134.0.149/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning ProbeError pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Readiness probe error: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused... 48m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Container created 48m Normal Killing pod/e2e-api-b-b5ea88-openshift-default-845d49979c-hcxzc Stopping container istio-proxy 48m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 Container started 48m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 Container created 48m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt Container started 48m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Add eth0 [10.133.0.209/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Readiness probe failed: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused 48m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Container started 48m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt Container created 48m Normal AddedLabel gateway/e2e-inf-a-34bcc5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-34bcc5 48m Normal SuccessfulCreate replicaset/e2e-inf-a-34bcc5-openshift-default-74b75878bc Created pod: e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr 48m Normal ScalingReplicaSet deployment/e2e-inf-a-34bcc5-openshift-default Scaled up replica set e2e-inf-a-34bcc5-openshift-default-74b75878bc from 0 to 1 48m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Container created 48m Normal Created pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Container created 48m Normal Started pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Container started 48m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Container started 48m Normal AddedInterface pod/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr Add eth0 [10.133.0.210/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-d15602 Scaled up replica set payload-pre-processing-e2e-inf-a-d15602-86cf9656c5 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-d15602 Scaled up replica set payload-processing-e2e-inf-a-d15602-5f9889cf64 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5 Error creating: pods "payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d15602: serviceaccount "payload-processing-e2e-inf-a-d15602" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-d15602-5f9889cf64 Error creating: pods "payload-processing-e2e-inf-a-d15602-5f9889cf64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d15602: serviceaccount "payload-processing-e2e-inf-a-d15602" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5 Created pod: payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-d15602-5f9889cf64 Created pod: payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr Add eth0 [10.134.0.150/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr Container started 48m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt Startup probe error: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt Startup probe failed: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused 48m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 48m Normal Pulled pod/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr Container created 48m Normal Pulled pod/payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 3; reason: All metrics below target 48m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx 48m Normal Created pod/payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r Container created 48m Normal AddedInterface pod/payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r Add eth0 [10.133.0.211/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Startup probe failed: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused 48m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default Scaled down replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 4 to 3 48m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Startup probe error: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused... 48m Normal Started pod/payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r Container started 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76 Error creating: pods "payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-34bcc5: serviceaccount "payload-processing-e2e-inf-a-34bcc5" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-34bcc5 Scaled up replica set payload-pre-processing-e2e-inf-a-34bcc5-556d888b55 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55 Created pod: payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55 Error creating: pods "payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-34bcc5: serviceaccount "payload-processing-e2e-inf-a-34bcc5" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76 Created pod: payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-r5frp 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-34bcc5 Scaled up replica set payload-processing-e2e-inf-a-34bcc5-66cfd9bc76 from 0 to 1 47m Warning ProbeError pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Startup probe error: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Startup probe failed: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused 47m Normal Pulled pod/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-r5frp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-r5frp Add eth0 [10.133.0.213/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv Container started 47m Normal Started pod/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-r5frp Container started 47m Normal Created pod/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv Container created 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-r5frp Container created 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv Add eth0 [10.133.0.212/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Readiness probe failed: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-qcqdx Readiness probe error: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal Killing pod/payload-processing-e2e-api-a-4bf33a-6bb59459b-4qdp6 Stopping container payload-processing 47m Normal Killing pod/payload-pre-processing-e2e-api-a-4bf33a-7f5fb57b55-9mjfq Stopping container payload-pre-processing 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-648d5f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-648d5f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal SuccessfulCreate replicaset/e2e-inf-b-a9542c-openshift-default-8c5766f89 Created pod: e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r 47m Normal AddedLabel gateway/e2e-inf-b-a9542c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-a9542c 47m Normal ScalingReplicaSet deployment/e2e-inf-b-a9542c-openshift-default Scaled up replica set e2e-inf-b-a9542c-openshift-default-8c5766f89 from 0 to 1 47m Normal AddedInterface pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Add eth0 [10.133.0.214/23] from ovn-kubernetes 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-a9542c-69bcb5578b Error creating: pods "payload-processing-e2e-inf-b-a9542c-69bcb5578b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-a9542c: serviceaccount "payload-processing-e2e-inf-b-a9542c" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-a9542c-794b65877f Created pod: payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-a9542c Scaled up replica set payload-processing-e2e-inf-b-a9542c-69bcb5578b from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-a9542c Scaled up replica set payload-pre-processing-e2e-inf-b-a9542c-794b65877f from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-a9542c-794b65877f Error creating: pods "payload-pre-processing-e2e-inf-b-a9542c-794b65877f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-a9542c: serviceaccount "payload-processing-e2e-inf-b-a9542c" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-a9542c-69bcb5578b Created pod: payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q 47m Normal Pulled pod/payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q Container started 47m Normal Created pod/payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q Container created 47m Normal AddedInterface pod/payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q Add eth0 [10.132.0.119/23] from ovn-kubernetes 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 Add eth0 [10.132.0.118/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 Container created 47m Normal Started pod/payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 Container started 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-ea9ed2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-ea9ed2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal Killing pod/e2e-api-b-1c4b82-openshift-default-7f9f99f747-ppqbt Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/e2e-inf-a-f3c580-openshift-default-5856c7d869 Created pod: e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 47m Normal AddedLabel gateway/e2e-inf-a-f3c580 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-f3c580 47m Normal ScalingReplicaSet deployment/e2e-inf-a-f3c580-openshift-default Scaled up replica set e2e-inf-a-f3c580-openshift-default-5856c7d869 from 0 to 1 47m Normal Killing pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/e2e-inf-a-409364-openshift-default-868c776847 Created pod: e2e-inf-a-409364-openshift-default-868c776847-ntjrp 47m Normal ScalingReplicaSet deployment/e2e-inf-a-409364-openshift-default Scaled up replica set e2e-inf-a-409364-openshift-default-868c776847 from 0 to 1 47m Normal AddedInterface pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 Add eth0 [10.133.0.216/23] from ovn-kubernetes 47m Normal AddedLabel gateway/e2e-inf-a-409364 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-409364 47m Normal AddedInterface pod/e2e-inf-a-409364-openshift-default-868c776847-ntjrp Add eth0 [10.133.0.217/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-94l6z Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-6lsmd Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-9kfm8 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-j46pc Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-nxs77 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning Unhealthy pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-api-b-2a2835-openshift-default-74dc4b7c9c-tvnbr Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-648d5f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-648d5f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal ScalingReplicaSet deployment/e2e-api-b-ea9ed2-openshift-default Scaled up replica set e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 from 4 to 8 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-f3c580-6d587b47f8 Error creating: pods "payload-processing-e2e-inf-a-f3c580-6d587b47f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f3c580: serviceaccount "payload-processing-e2e-inf-a-f3c580" not found 47m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn 47m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-f3c580 Scaled up replica set payload-pre-processing-e2e-inf-a-f3c580-7bff98f799 from 0 to 1 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-ea9ed2-openshift-default New size: 8; reason: All metrics below target 47m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 47m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-f3c580 Scaled up replica set payload-processing-e2e-inf-a-f3c580-6d587b47f8 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-f3c580-6d587b47f8 Created pod: payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799 Created pod: payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799 Error creating: pods "payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f3c580: serviceaccount "payload-processing-e2e-inf-a-f3c580" not found 47m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Container started 47m Normal Started pod/payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw Container started 47m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Add eth0 [10.134.0.151/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Container created 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q Add eth0 [10.132.0.121/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q Container started 47m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Add eth0 [10.133.0.219/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Container started 47m Normal Created pod/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q Container created 47m Normal Created pod/payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw Container created 47m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Container started 47m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Container created 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw Add eth0 [10.132.0.122/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Container created 47m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Add eth0 [10.134.0.152/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn Add eth0 [10.132.0.120/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn Container created 47m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn Container started 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-409364-89dc78747 Created pod: payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-409364-58bb5dbb9b Error creating: pods "payload-processing-e2e-inf-a-409364-58bb5dbb9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-409364: serviceaccount "payload-processing-e2e-inf-a-409364" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-409364-89dc78747 Error creating: pods "payload-pre-processing-e2e-inf-a-409364-89dc78747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-409364: serviceaccount "payload-processing-e2e-inf-a-409364" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-409364 Scaled up replica set payload-pre-processing-e2e-inf-a-409364-89dc78747 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-409364 Scaled up replica set payload-processing-e2e-inf-a-409364-58bb5dbb9b from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-409364-58bb5dbb9b Created pod: payload-processing-e2e-inf-a-409364-58bb5dbb9b-rdzd5 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-409364-58bb5dbb9b-rdzd5 Add eth0 [10.132.0.123/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh Container started 47m Normal Pulled pod/payload-processing-e2e-inf-a-409364-58bb5dbb9b-rdzd5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-inf-a-409364-58bb5dbb9b-rdzd5 Container created 47m Normal Started pod/payload-processing-e2e-inf-a-409364-58bb5dbb9b-rdzd5 Container started 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh Add eth0 [10.134.0.153/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh Container created 47m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled down replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 4 to 3 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default New size: 3; reason: All metrics below target 47m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Startup probe failed: Get "http://10.133.0.219:15021/healthz/ready": dial tcp 10.133.0.219:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Startup probe error: Get "http://10.133.0.219:15021/healthz/ready": dial tcp 10.133.0.219:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-292pb Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 (combined from similar events): Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf 47m Normal SuccessfulCreate replicaset/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 Created pod: e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-ea9ed2-openshift-default New size: 10; reason: All metrics below target 47m Normal ScalingReplicaSet deployment/e2e-api-b-ea9ed2-openshift-default Scaled up replica set e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9 from 8 to 10 47m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Container started 47m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Container created 47m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Add eth0 [10.132.0.124/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Add eth0 [10.133.0.221/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Container created 47m Normal Started pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Container started 47m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Startup probe failed: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Startup probe error: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-6gclk Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-6gclk 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 10 to 7 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-7mtkg 46m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-swl4n Stopping container istio-proxy 46m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-7mtkg Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-swl4n 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 46m Normal Killing pod/e2e-api-a-4bf33a-openshift-default-b849dbcc5-ttvq9 Stopping container istio-proxy 46m Normal Killing pod/payload-processing-e2e-api-b-ea9ed2-78fc9fb9fb-ftnvk Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-api-b-ea9ed2-849b7b59dc-vp88x Stopping container payload-pre-processing 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl 46m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Stopping container istio-proxy 46m Normal SuccessfulCreate replicaset/e2e-inf-b-93549e-openshift-default-5c77df5c4c Created pod: e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 46m Normal AddedLabel gateway/e2e-inf-b-93549e Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-93549e 46m Normal ScalingReplicaSet deployment/e2e-inf-b-93549e-openshift-default Scaled up replica set e2e-inf-b-93549e-openshift-default-5c77df5c4c from 0 to 1 46m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-g6hjl Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal AddedInterface pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Add eth0 [10.133.0.225/23] from ovn-kubernetes 46m Normal Started pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Container started 46m Normal Created pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Container created 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-a9542c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Pulled pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-a9542c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Pulled pod/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Container created 46m Normal Pulled pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr Container started 46m Normal Created pod/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr Container created 46m Normal Started pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Container started 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-34bcc5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-34bcc5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning Unhealthy pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Startup probe failed: Get "http://10.133.0.207:15021/healthz/ready": dial tcp 10.133.0.207:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Startup probe failed: Get "http://10.133.0.214:15021/healthz/ready": dial tcp 10.133.0.214:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Startup probe error: Get "http://10.133.0.214:15021/healthz/ready": dial tcp 10.133.0.214:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Startup probe error: Get "http://10.133.0.207:15021/healthz/ready": dial tcp 10.133.0.207:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr Startup probe failed: Get "http://10.133.0.210:15021/healthz/ready": dial tcp 10.133.0.210:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr Startup probe error: Get "http://10.133.0.210:15021/healthz/ready": dial tcp 10.133.0.210:15021: connect: connection refused... 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-93549e-779fdd989f Created pod: payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-93549e-56ff978bf6 Error creating: pods "payload-processing-e2e-inf-b-93549e-56ff978bf6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-93549e: serviceaccount "payload-processing-e2e-inf-b-93549e" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-93549e Scaled up replica set payload-processing-e2e-inf-b-93549e-56ff978bf6 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-93549e Scaled up replica set payload-pre-processing-e2e-inf-b-93549e-779fdd989f from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-93549e-56ff978bf6 Created pod: payload-processing-e2e-inf-b-93549e-56ff978bf6-qh2c5 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-93549e-779fdd989f Error creating: pods "payload-pre-processing-e2e-inf-b-93549e-779fdd989f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-93549e: serviceaccount "payload-processing-e2e-inf-b-93549e" not found 46m Normal Started pod/payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf Container started 46m Normal Created pod/payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf Container created 46m Normal Pulled pod/payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-inf-b-93549e-56ff978bf6-qh2c5 Container started 46m Normal Created pod/payload-processing-e2e-inf-b-93549e-56ff978bf6-qh2c5 Container created 46m Normal Pulled pod/payload-processing-e2e-inf-b-93549e-56ff978bf6-qh2c5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf Add eth0 [10.134.0.154/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-processing-e2e-inf-b-93549e-56ff978bf6-qh2c5 Add eth0 [10.132.0.126/23] from ovn-kubernetes 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d15602-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d15602-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-hswld 45m Normal Created pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 Container created 45m Normal Started pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 Container started 45m Normal Pulled pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/e2e-inf-a-409364-openshift-default-868c776847-ntjrp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-inf-a-409364-openshift-default-868c776847-ntjrp Container created 45m Normal Started pod/e2e-inf-a-409364-openshift-default-868c776847-ntjrp Container started 45m Normal AddedLabel gateway/e2e-inf-b-fde75d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-fde75d 45m Normal ScalingReplicaSet deployment/e2e-inf-b-0b1c15-openshift-default Scaled up replica set e2e-inf-b-0b1c15-openshift-default-7d89c8877c from 0 to 1 45m Normal ScalingReplicaSet deployment/e2e-inf-b-fde75d-openshift-default Scaled up replica set e2e-inf-b-fde75d-openshift-default-6fb6875ff from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-inf-b-0b1c15-openshift-default-7d89c8877c Created pod: e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh 45m Normal AddedLabel gateway/e2e-inf-b-0b1c15 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-0b1c15 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-a9542c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning Unhealthy pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 Startup probe failed: Get "http://10.133.0.216:15021/healthz/ready": dial tcp 10.133.0.216:15021: connect: connection refused 45m Normal AddedInterface pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh Add eth0 [10.132.0.127/23] from ovn-kubernetes 45m Warning Unhealthy pod/e2e-inf-a-409364-openshift-default-868c776847-ntjrp Startup probe failed: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused 45m Warning ProbeError pod/e2e-inf-a-409364-openshift-default-868c776847-ntjrp Startup probe error: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused... 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-a9542c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning ProbeError pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 Startup probe error: Get "http://10.133.0.216:15021/healthz/ready": dial tcp 10.133.0.216:15021: connect: connection refused... 45m Normal SuccessfulCreate replicaset/e2e-inf-b-fde75d-openshift-default-6fb6875ff Created pod: e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl 45m Normal AddedInterface pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl Add eth0 [10.133.0.227/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-34bcc5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-34bcc5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74 Error creating: pods "payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0b1c15: serviceaccount "payload-processing-e2e-inf-b-0b1c15" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-0b1c15-754948978c Error creating: pods "payload-pre-processing-e2e-inf-b-0b1c15-754948978c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0b1c15: serviceaccount "payload-processing-e2e-inf-b-0b1c15" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-0b1c15 Scaled up replica set payload-processing-e2e-inf-b-0b1c15-5bccfc8c74 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74 Created pod: payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-0b1c15 Scaled up replica set payload-pre-processing-e2e-inf-b-0b1c15-754948978c from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-0b1c15-754948978c Created pod: payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n 45m Normal AddedLabel gateway/e2e-inf-b-b9c4b4 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-b9c4b4 45m Normal Created pod/payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n Container created 45m Normal Started pod/payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n Container started 45m Normal Pulled pod/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 Container created 45m Normal Started pod/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 Container started 45m Normal SuccessfulCreate replicaset/e2e-inf-b-b9c4b4-openshift-default-756965fdd5 Created pod: e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n Add eth0 [10.134.0.155/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 Add eth0 [10.134.0.156/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/e2e-inf-b-b9c4b4-openshift-default Scaled up replica set e2e-inf-b-b9c4b4-openshift-default-756965fdd5 from 0 to 1 45m Normal AddedInterface pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Add eth0 [10.133.0.228/23] from ovn-kubernetes 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f3c580-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-fde75d Scaled up replica set payload-pre-processing-e2e-inf-b-fde75d-75cd788695 from 0 to 1 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f3c580-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb Error creating: pods "payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-fde75d: serviceaccount "payload-processing-e2e-inf-b-fde75d" not found 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb Created pod: payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-fde75d-75cd788695 Error creating: pods "payload-pre-processing-e2e-inf-b-fde75d-75cd788695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-fde75d: serviceaccount "payload-processing-e2e-inf-b-fde75d" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-fde75d Scaled up replica set payload-processing-e2e-inf-b-fde75d-7d8d99ffcb from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-fde75d-75cd788695 Created pod: payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq 45m Normal Started pod/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 Container started 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq Add eth0 [10.132.0.128/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq Container created 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 Add eth0 [10.132.0.129/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 Container created 45m Normal Started pod/payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq Container started 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-b9c4b4-76948497d Error creating: pods "payload-processing-e2e-inf-b-b9c4b4-76948497d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-b9c4b4: serviceaccount "payload-processing-e2e-inf-b-b9c4b4" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-b9c4b4 Scaled up replica set payload-processing-e2e-inf-b-b9c4b4-76948497d from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-b9c4b4 Scaled up replica set payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-b9c4b4-76948497d Created pod: payload-processing-e2e-inf-b-b9c4b4-76948497d-7vhrd 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67 Error creating: pods "payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-b9c4b4: serviceaccount "payload-processing-e2e-inf-b-b9c4b4" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67 Created pod: payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq Add eth0 [10.132.0.131/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-inf-b-b9c4b4-76948497d-7vhrd Container started 45m Normal Created pod/payload-processing-e2e-inf-b-b9c4b4-76948497d-7vhrd Container created 45m Normal Pulled pod/payload-processing-e2e-inf-b-b9c4b4-76948497d-7vhrd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-b9c4b4-76948497d-7vhrd Add eth0 [10.132.0.132/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq Container started 45m Normal Created pod/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq Container created 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f3c580-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f3c580-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-409364-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-409364-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d15602-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d15602-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-fde75d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-fde75d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Stopping container istio-proxy 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-qp4zn Stopping container istio-proxy 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Stopping container istio-proxy 45m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-vldc5 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-tbpv5 Stopping container istio-proxy 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-4rgtt Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Stopping container istio-proxy 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-9crds Stopping container istio-proxy 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Killing pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-x7wtf Stopping container istio-proxy 45m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-t8rbm Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal ScalingReplicaSet deployment/e2e-inf-a-019e5b-openshift-default Scaled up replica set e2e-inf-a-019e5b-openshift-default-7d7cb97b95 from 0 to 1 44m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-wcsg7 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal SuccessfulCreate replicaset/e2e-inf-a-019e5b-openshift-default-7d7cb97b95 Created pod: e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc 44m Normal AddedLabel gateway/e2e-inf-a-019e5b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-019e5b 44m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-djct8 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal AddedInterface pod/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc Add eth0 [10.134.0.158/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal Started pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Container started 44m Normal Created pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Container created 44m Normal Pulled pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning Unhealthy pod/e2e-api-b-ea9ed2-openshift-default-6b9bddc5c9-zrtbg Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal Killing pod/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx Stopping container payload-processing 44m Normal Killing pod/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh Stopping container payload-pre-processing 44m Warning ProbeError pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Startup probe error: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Startup probe failed: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-019e5b-6578bd7576 Created pod: payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755 Error creating: pods "payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-019e5b: serviceaccount "payload-processing-e2e-inf-a-019e5b" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755 Created pod: payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-019e5b Scaled up replica set payload-pre-processing-e2e-inf-a-019e5b-75f68bd755 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-019e5b Scaled up replica set payload-processing-e2e-inf-a-019e5b-6578bd7576 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-019e5b-6578bd7576 Error creating: pods "payload-processing-e2e-inf-a-019e5b-6578bd7576-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-019e5b: serviceaccount "payload-processing-e2e-inf-a-019e5b" not found 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl Add eth0 [10.134.0.159/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh Add eth0 [10.132.0.135/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh Container created 44m Normal Started pod/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh Container started 44m Normal Pulled pod/payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl Container created 44m Normal Started pod/payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl Container started 44m Warning Unhealthy pod/payload-processing-e2e-inf-a-5a9d7b-6f7494c4b8-flvrx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 44m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-5a9d7b-9bb744df9-2qrjh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 44m Normal Killing pod/e2e-inf-a-5a9d7b-openshift-default-796bc9cccf-z2m7t Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 3 to 2 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 2; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 2; reason: All metrics below target 44m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled down replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 3 to 2 44m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-k5gml Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-93549e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-93549e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-7qc25 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal Killing pod/payload-processing-e2e-inf-b-ca02c9-7fc799d5c8-pkcxp Stopping container payload-processing 44m Normal Killing pod/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h Stopping container payload-pre-processing 44m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-wn5g6 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default New size: 2; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h 44m Normal Pulled pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh Container created 44m Normal Started pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh Container started 44m Normal Pulled pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl Container created 44m Normal Started pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl Container started 44m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-vg97h Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning ProbeError pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0b1c15-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0b1c15-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning ProbeError pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl Startup probe error: Get "http://10.133.0.227:15021/healthz/ready": dial tcp 10.133.0.227:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl Startup probe failed: Get "http://10.133.0.227:15021/healthz/ready": dial tcp 10.133.0.227:15021: connect: connection refused 44m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-ca02c9-594d5d6fff-mvr9h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0b1c15-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0b1c15-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-fde75d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-fde75d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Readiness probe failed: Get "http://10.133.0.228:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Readiness probe error: Get "http://10.133.0.228:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Pulled pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Container created 43m Normal Started pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Container started 43m Warning ProbeError pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Startup probe error: Get "http://10.133.0.228:15021/healthz/ready": dial tcp 10.133.0.228:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Startup probe failed: Get "http://10.133.0.228:15021/healthz/ready": dial tcp 10.133.0.228:15021: connect: connection refused 43m Normal Killing pod/e2e-inf-b-ca02c9-openshift-default-67db76697-d8mwv Stopping container istio-proxy 43m Normal AddedLabel gateway/e2e-shared-a-w5-acefbb Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-acefbb 43m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-acefbb-openshift-default Scaled up replica set e2e-shared-a-w5-acefbb-openshift-default-7c99445d75 from 0 to 1 43m Normal AddedInterface pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Add eth0 [10.133.0.238/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75 Created pod: e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-b9c4b4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-b9c4b4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-acefbb-569588ff76 Created pod: payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd 43m Normal Created pod/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc Container created 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-acefbb Scaled up replica set payload-processing-e2e-shared-a-w5-acefbb-569588ff76 from 0 to 1 43m Normal Started pod/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc Container started 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-acefbb: serviceaccount "payload-processing-e2e-shared-a-w5-acefbb" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47 Created pod: payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk 43m Normal Pulled pod/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-acefbb Scaled up replica set payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-acefbb-569588ff76 Error creating: pods "payload-processing-e2e-shared-a-w5-acefbb-569588ff76-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-acefbb: serviceaccount "payload-processing-e2e-shared-a-w5-acefbb" not found 43m Normal Started pod/payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd Container started 43m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd Add eth0 [10.134.0.161/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk Add eth0 [10.134.0.160/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk Container started 43m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk Container created 43m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning Unhealthy pod/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc Startup probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc Startup probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-b9c4b4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-b9c4b4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-f3c580-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/e2e-inf-a-f3c580-openshift-default Scaled up replica set e2e-inf-a-f3c580-openshift-default-5856c7d869 from 1 to 2 43m Normal AddedInterface pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t Add eth0 [10.134.0.162/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/e2e-inf-a-f3c580-openshift-default-5856c7d869 Created pod: e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-019e5b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Pulled pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t Container created 43m Normal Started pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t Container started 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-019e5b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/e2e-inf-b-93549e-openshift-default Scaled up replica set e2e-inf-b-93549e-openshift-default-5c77df5c4c from 1 to 2 43m Normal Pulled pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal SuccessfulCreate replicaset/e2e-inf-b-93549e-openshift-default-5c77df5c4c Created pod: e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm 43m Normal AddedInterface pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Add eth0 [10.134.0.163/23] from ovn-kubernetes 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-93549e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal Started pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Container started 43m Normal Created pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Container created 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-d15602-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/e2e-inf-a-d15602-openshift-default Scaled up replica set e2e-inf-a-d15602-openshift-default-6b94954795 from 1 to 2 43m Normal SuccessfulCreate replicaset/e2e-inf-a-d15602-openshift-default-6b94954795 Created pod: e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj 43m Normal AddedInterface pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Add eth0 [10.134.0.164/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 43m Normal Pulled pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Container created 43m Normal Started pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Container started 43m Warning ProbeError pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Startup probe error: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Startup probe failed: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Startup probe error: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Startup probe failed: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused 43m Normal SuccessfulCreate replicaset/e2e-inf-b-1d3696-openshift-default-8769f4c5f Created pod: e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w 43m Normal AddedLabel gateway/e2e-inf-b-1d3696 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-1d3696 43m Normal ScalingReplicaSet deployment/e2e-inf-b-1d3696-openshift-default Scaled up replica set e2e-inf-b-1d3696-openshift-default-8769f4c5f from 0 to 1 43m Normal AddedInterface pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w Add eth0 [10.133.0.240/23] from ovn-kubernetes 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-019e5b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-019e5b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-1d3696 Scaled up replica set payload-processing-e2e-inf-b-1d3696-b6fbdfb88 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-1d3696 Scaled up replica set payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5 Created pod: payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-1d3696-b6fbdfb88 Created pod: payload-processing-e2e-inf-b-1d3696-b6fbdfb88-wdqxq 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-1d3696-b6fbdfb88 Error creating: pods "payload-processing-e2e-inf-b-1d3696-b6fbdfb88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1d3696: serviceaccount "payload-processing-e2e-inf-b-1d3696" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5 Error creating: pods "payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1d3696: serviceaccount "payload-processing-e2e-inf-b-1d3696" not found 42m Normal AddedInterface pod/payload-processing-e2e-inf-b-1d3696-b6fbdfb88-wdqxq Add eth0 [10.134.0.165/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-inf-b-1d3696-b6fbdfb88-wdqxq Container created 42m Normal Pulled pod/payload-processing-e2e-inf-b-1d3696-b6fbdfb88-wdqxq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-inf-b-1d3696-b6fbdfb88-wdqxq Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k Add eth0 [10.132.0.141/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k Container created 42m Normal Started pod/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k Container started 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-jvvbq 42m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 7 to 6 42m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-jvvbq Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default New size: 2; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default Scaled down replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 3 to 2 42m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Stopping container istio-proxy 42m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-frblp Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-x52tn Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal Started pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt Container started 42m Normal Pulled pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-0b1c15-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal Created pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt Container created 42m Normal SuccessfulCreate replicaset/e2e-inf-b-0b1c15-openshift-default-7d89c8877c Created pod: e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt 42m Normal ScalingReplicaSet deployment/e2e-inf-b-0b1c15-openshift-default Scaled up replica set e2e-inf-b-0b1c15-openshift-default-7d89c8877c from 1 to 2 42m Normal AddedInterface pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt Add eth0 [10.134.0.167/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused 42m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 42m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-56mz2 Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 6 to 5 42m Normal Killing pod/payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 Stopping container payload-processing 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-56mz2 42m Normal Killing pod/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc Stopping container payload-pre-processing 41m Warning Unhealthy pod/payload-processing-e2e-inf-a-648d5f-54b7c56458-79pb4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-648d5f-7fc79c5b59-j4bjc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Warning ProbeError pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Readiness probe error: Get "http://10.133.0.238:15021/healthz/ready": read tcp 10.133.0.2:48946->10.133.0.238:15021: read: connection reset by peer... 41m Warning Unhealthy pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Readiness probe failed: Get "http://10.133.0.238:15021/healthz/ready": read tcp 10.133.0.2:48946->10.133.0.238:15021: read: connection reset by peer 41m Normal Created pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Container created 41m Normal Pulled pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Container started 41m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-1d6347-openshift-default Scaled up replica set e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f from 0 to 1 41m Normal AddedLabel gateway/e2e-shared-b-w5-1d6347 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-1d6347 41m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f Created pod: e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 41m Normal AddedInterface pod/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 Add eth0 [10.132.0.145/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Startup probe failed: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Startup probe error: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused... 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d Created pod: payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d Error creating: pods "payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-1d6347: serviceaccount "payload-processing-e2e-shared-b-w5-1d6347" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-1d6347: serviceaccount "payload-processing-e2e-shared-b-w5-1d6347" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655 Created pod: payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-1d6347 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-1d6347 Scaled up replica set payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d from 0 to 1 41m Normal Pulled pod/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn Container started 41m Normal Created pod/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 Container created 41m Normal Started pod/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 Container started 41m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 Add eth0 [10.134.0.168/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn Add eth0 [10.132.0.146/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn Container created 41m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 5 to 4 41m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-hszvg 41m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Stopping container istio-proxy 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-acefbb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-acefbb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal Killing pod/e2e-inf-a-648d5f-openshift-default-86d675f7b9-dr4nh Stopping container istio-proxy 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-hszvg Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal Killing pod/payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q Stopping container payload-processing 40m Normal Pulled pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w Container started 40m Normal Created pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w Container created 40m Normal ScalingReplicaSet deployment/e2e-inf-b-fde75d-openshift-default Scaled up replica set e2e-inf-b-fde75d-openshift-default-6fb6875ff from 1 to 2 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-fde75d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-inf-b-fde75d-openshift-default-6fb6875ff Created pod: e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq 40m Normal Started pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Container started 40m Normal Created pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Container created 40m Normal Pulled pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Add eth0 [10.134.0.181/23] from ovn-kubernetes 40m Warning Unhealthy pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w Startup probe failed: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w Startup probe error: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Startup probe error: Get "http://10.134.0.181:15021/healthz/ready": dial tcp 10.134.0.181:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Startup probe failed: Get "http://10.134.0.181:15021/healthz/ready": dial tcp 10.134.0.181:15021: connect: connection refused 40m Warning Unhealthy pod/payload-processing-e2e-inf-b-a9542c-69bcb5578b-rzp2q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-a9542c-794b65877f-sp225 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-wjqps 40m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 40m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 40m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-wjqps Stopping container discovery 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r Stopping container payload-processing 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-1d3696-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-1d3696-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Stopping container istio-proxy 40m Warning ProbeError pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65 Created pod: e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj 40m Warning Unhealthy pod/e2e-inf-b-a9542c-openshift-default-8c5766f89-tsg6r Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-396d70-openshift-default Scaled up replica set e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65 from 0 to 1 40m Normal AddedLabel gateway/e2e-shared-a-w2-396d70 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-396d70 40m Normal AddedInterface pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Add eth0 [10.132.0.162/23] from ovn-kubernetes 40m Normal Created pod/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 Container created 40m Normal Pulled pod/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 Container started 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-d15602-86cf9656c5-cr7rr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975 Created pod: payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6 Created pod: payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-396d70 Scaled up replica set payload-processing-e2e-shared-a-w2-396d70-86964fb4d6 from 0 to 1 39m Warning Unhealthy pod/payload-processing-e2e-inf-a-d15602-5f9889cf64-z4t2r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-396d70: serviceaccount "payload-processing-e2e-shared-a-w2-396d70" not found 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6 Error creating: pods "payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-396d70: serviceaccount "payload-processing-e2e-shared-a-w2-396d70" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-396d70 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975 from 0 to 1 39m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 Container started 39m Normal Pulled pod/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm Container started 39m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 Container created 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 Add eth0 [10.132.0.165/23] from ovn-kubernetes 39m Warning Unhealthy pod/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 Startup probe failed: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused 39m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning ProbeError pod/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 Startup probe error: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused... 39m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm Add eth0 [10.132.0.164/23] from ovn-kubernetes 39m Normal Created pod/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm Container created 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-1d6347-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-1d6347-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Stopping container istio-proxy 39m Warning Unhealthy pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-inf-a-d15602-openshift-default-6b94954795-nvxwj Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Killing pod/payload-processing-e2e-inf-a-34bcc5-66cfd9bc76-r5frp Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv Stopping container payload-pre-processing 39m Warning Unhealthy pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh Stopping container payload-pre-processing 39m Warning ProbeError pod/e2e-inf-a-d15602-openshift-default-6b94954795-cpt8d Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Killing pod/payload-processing-e2e-inf-a-409364-58bb5dbb9b-rdzd5 Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-inf-b-93549e-56ff978bf6-qh2c5 Stopping container payload-processing 39m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-00926c-openshift-default-5b8b9f5575 Deleted pod: e2e-worker-w3-00926c-openshift-default-5b8b9f5575-kg9dv 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-00926c-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-00926c-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-00926c-openshift-default-5b8b9f5575 from 2 to 1 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-409364-89dc78747-j29bh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-34bcc5-556d888b55-7vxmv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Stopping container istio-proxy 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-93549e-779fdd989f-7cpgf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning ProbeError pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-jx8k8 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-inf-b-93549e-openshift-default-5c77df5c4c-pb7fm Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal AddedInterface pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z Add eth0 [10.132.0.171/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-inf-b-1d3696-openshift-default-8769f4c5f Created pod: e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-1d3696-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/e2e-inf-b-1d3696-openshift-default Scaled up replica set e2e-inf-b-1d3696-openshift-default-8769f4c5f from 1 to 2 39m Normal Pulled pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z Container created 39m Normal Started pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z Container started 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-1d6347-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-1d6347-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning Unhealthy pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 38m Normal Killing pod/e2e-inf-a-34bcc5-openshift-default-74b75878bc-ddnhr Stopping container istio-proxy 38m Normal Killing pod/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw Stopping container payload-processing 38m Normal Killing pod/e2e-inf-a-409364-openshift-default-868c776847-ntjrp Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Readiness probe failed: Get "http://10.132.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Readiness probe error: Get "http://10.132.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Killing pod/payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 Stopping container payload-processing 38m Normal Pulled pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Container created 38m Normal Started pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Container started 38m Normal AddedInterface pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 Add eth0 [10.132.0.175/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-acefbb-openshift-default Scaled up replica set e2e-shared-a-w5-acefbb-openshift-default-7c99445d75 from 1 to 2 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-acefbb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75 Created pod: e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 38m Warning ProbeError pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 38m Normal Created pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 Container created 38m Normal Started pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 Container started 38m Warning Unhealthy pod/payload-processing-e2e-inf-a-f3c580-6d587b47f8-zthxw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal Killing pod/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-inf-b-b9c4b4-76948497d-7vhrd Stopping container payload-processing 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-f3c580-7bff98f799-c6d7q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-396d70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-396d70-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning Unhealthy pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 Startup probe failed: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 Startup probe error: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused... 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-6e91df-openshift-default Scaled up replica set e2e-shared-b-w2-6e91df-openshift-default-8689d4c967 from 0 to 1 38m Normal AddedLabel gateway/e2e-shared-b-w2-6e91df Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-6e91df 38m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 4 to 3 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967 Created pod: e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-fde75d-75cd788695-5n8hq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-5842c 38m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 38m Warning Unhealthy pod/payload-processing-e2e-inf-b-fde75d-7d8d99ffcb-79hw8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal AddedInterface pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc Add eth0 [10.133.0.252/23] from ovn-kubernetes 38m Normal Killing pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-4cx4t Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-a-f3c580-openshift-default-5856c7d869-rq7j6 Stopping container istio-proxy 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Readiness probe failed: Get "http://10.132.0.39:15021/healthz/ready": dial tcp 10.132.0.39:15021: connect: connection refused 38m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-5842c Readiness probe error: Get "http://10.132.0.39:15021/healthz/ready": dial tcp 10.132.0.39:15021: connect: connection refused... 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-6e91df Scaled up replica set payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-6e91df: serviceaccount "payload-processing-e2e-shared-b-w2-6e91df" not found 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-6e91df Scaled up replica set payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4 Created pod: payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675 Created pod: payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4 Error creating: pods "payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-6e91df: serviceaccount "payload-processing-e2e-shared-b-w2-6e91df" not found 38m Normal Created pod/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c Container created 38m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal ScalingReplicaSet deployment/e2e-worker-w5-a7bb0c-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf from 2 to 1 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-a7bb0c-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-ca229c-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w1-ca229c-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-ca229c-openshift-default-549cb656c9 from 2 to 1 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-ca229c-openshift-default-549cb656c9 Deleted pod: e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp 38m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g Container created 38m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Stopping container istio-proxy 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g Add eth0 [10.133.0.253/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c Container started 38m Normal SuccessfulDelete replicaset/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf Deleted pod: e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 38m Normal Pulled pod/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g Container started 38m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c Add eth0 [10.133.0.254/23] from ovn-kubernetes 38m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-t5xdp Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-e371c3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e371c3-openshift-default-699697d895 Created pod: e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e371c3-openshift-default Scaled up replica set e2e-shared-a-w4-e371c3-openshift-default-699697d895 from 1 to 2 38m Normal Started pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr Container started 38m Normal Created pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr Container created 38m Normal Pulled pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr Add eth0 [10.132.0.178/23] from ovn-kubernetes 38m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-zdx74 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal Killing pod/payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 Stopping container payload-processing 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-b9c4b4-6498846c67-q52pq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning ProbeError pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr Startup probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr Startup probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused 38m Normal Killing pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-9dhnl Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w0-2df0ab-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-2df0ab-openshift-default-757d5db88c from 2 to 1 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2df0ab-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w0-2df0ab-openshift-default-757d5db88c Deleted pod: e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg 37m Warning ProbeError pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-inf-b-fde75d-openshift-default-6fb6875ff-khlmq Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-396d70-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-396d70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning ProbeError pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-kwzdg Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Stopping container istio-proxy 37m Warning Unhealthy pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-inf-b-b9c4b4-openshift-default-756965fdd5-jz5hk Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/payload-processing-e2e-inf-b-0b1c15-5bccfc8c74-b7c65 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-0b1c15-754948978c-wql6n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f Created pod: e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-9e3aa0-openshift-default Scaled up replica set e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f from 0 to 1 37m Normal AddedLabel gateway/e2e-shared-a-w0-9e3aa0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-9e3aa0 37m Normal AddedInterface pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Add eth0 [10.133.1.1/23] from ovn-kubernetes 37m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-9e3aa0 Scaled up replica set payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6 from 0 to 1 37m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 3 to 2 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 37m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-dsk2f Container started 37m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-dsk2f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-dsk2f Add eth0 [10.133.1.2/23] from ovn-kubernetes 37m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-d2gh4 Stopping container istio-proxy 37m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-dsk2f Container created 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-9e3aa0: serviceaccount "payload-processing-e2e-shared-a-w0-9e3aa0" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599 Created pod: payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-dsk2f 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-9e3aa0 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6 Created pod: payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-qrkwf 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6 Error creating: pods "payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-9e3aa0: serviceaccount "payload-processing-e2e-shared-a-w0-9e3aa0" not found 37m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-qrkwf Add eth0 [10.133.1.3/23] from ovn-kubernetes 37m Normal Killing pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-48jxt Stopping container istio-proxy 37m Normal Pulled pod/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-qrkwf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-qrkwf Container created 37m Normal Started pod/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-qrkwf Container started 37m Normal Killing pod/e2e-inf-b-0b1c15-openshift-default-7d89c8877c-zpkwh Stopping container istio-proxy 37m Normal Killing pod/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl Stopping container payload-processing 37m Warning Unhealthy pod/payload-processing-e2e-inf-a-019e5b-6578bd7576-w9dfl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-019e5b-75f68bd755-f22gh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal Pulled pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc Container created 37m Normal Started pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc Container started 37m Warning Unhealthy pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc Startup probe failed: Get "http://10.133.0.252:15021/healthz/ready": dial tcp 10.133.0.252:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc Startup probe error: Get "http://10.133.0.252:15021/healthz/ready": dial tcp 10.133.0.252:15021: connect: connection refused... 36m Normal Killing pod/e2e-inf-a-019e5b-openshift-default-7d7cb97b95-cdrkc Stopping container istio-proxy 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-6e91df-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-6e91df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/payload-processing-e2e-inf-b-1d3696-b6fbdfb88-wdqxq Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k Stopping container payload-pre-processing 36m Normal Created pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Container created 36m Normal Started pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Container started 36m Normal Pulled pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-9e3aa0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-9e3aa0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning ProbeError pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Startup probe error: Get "http://10.133.1.1:15021/healthz/ready": dial tcp 10.133.1.1:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Startup probe failed: Get "http://10.133.1.1:15021/healthz/ready": dial tcp 10.133.1.1:15021: connect: connection refused 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-a59ecb-openshift-default Scaled up replica set e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 from 1 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-a59ecb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 Created pod: e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx 36m Normal Created pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx Container created 36m Normal Started pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx Container started 36m Normal AddedInterface pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx Add eth0 [10.132.0.184/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-1d3696-69b88b47c5-kpq6k Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx Startup probe failed: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx Startup probe error: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused... 36m Normal AddedLabel gateway/e2e-shared-b-w0-f1f206 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-f1f206 36m Normal AddedInterface pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Add eth0 [10.133.1.10/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-f1f206-openshift-default Scaled up replica set e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9 Created pod: e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v 36m Normal Pulled pod/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-kb9kl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff Created pod: payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-k76fk 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8 Created pod: payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-kb9kl 36m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-k76fk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-f1f206 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff from 0 to 1 36m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-kb9kl Add eth0 [10.134.0.187/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-k76fk Add eth0 [10.133.1.12/23] from ovn-kubernetes 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8 Error creating: pods "payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-f1f206: serviceaccount "payload-processing-e2e-shared-b-w0-f1f206" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-f1f206 Scaled up replica set payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8 from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff Error creating: pods "payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-f1f206: serviceaccount "payload-processing-e2e-shared-b-w0-f1f206" not found 36m Normal Created pod/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-kb9kl Container created 36m Normal Started pod/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-kb9kl Container started 35m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-k76fk Container created 35m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-k76fk Container started 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-9e3aa0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-9e3aa0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-ljm4w Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-b-1d3696-openshift-default-8769f4c5f-nkl9z Stopping container istio-proxy 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59 Created pod: e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-c7c37b-openshift-default Scaled up replica set e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59 from 0 to 1 35m Normal AddedLabel gateway/e2e-shared-a-w1-c7c37b Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-c7c37b 35m Normal AddedInterface pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl Add eth0 [10.133.1.13/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-c7c37b Scaled up replica set payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-c7c37b Scaled up replica set payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999 from 0 to 1 35m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr Container created 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999 Created pod: payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-c7c37b: serviceaccount "payload-processing-e2e-shared-a-w1-c7c37b" not found 35m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m Add eth0 [10.132.0.186/23] from ovn-kubernetes 35m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr Container started 35m Normal Pulled pod/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr Add eth0 [10.132.0.185/23] from ovn-kubernetes 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964 Created pod: payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964 Error creating: pods "payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-c7c37b: serviceaccount "payload-processing-e2e-shared-a-w1-c7c37b" not found 35m Normal Created pod/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m Container created 35m Normal Started pod/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m Container started 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-228454-openshift-default-6476dbf645 Created pod: e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-228454-openshift-default Scaled up replica set e2e-shared-b-w3-228454-openshift-default-6476dbf645 from 1 to 2 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-228454-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal Started pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s Container started 35m Normal AddedLabel gateway/e2e-ipp-a-d3a829 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-d3a829 35m Normal Created pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s Container created 35m Normal Pulled pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s Add eth0 [10.133.1.14/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx Add eth0 [10.133.1.16/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/e2e-ipp-a-d3a829-openshift-default Scaled up replica set e2e-ipp-a-d3a829-openshift-default-74d65df7df from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-ipp-a-d3a829-openshift-default-74d65df7df Created pod: e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx 35m Warning ProbeError pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s Startup probe error: Get "http://10.133.1.14:15021/healthz/ready": dial tcp 10.133.1.14:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s Startup probe failed: Get "http://10.133.1.14:15021/healthz/ready": dial tcp 10.133.1.14:15021: connect: connection refused 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547 Created pod: payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-xlcmh 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547 Error creating: pods "payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d3a829: serviceaccount "payload-processing-e2e-ipp-a-d3a829" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-d3a829 Scaled up replica set payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547 from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-d3a829-698854d99d Error creating: pods "payload-processing-e2e-ipp-a-d3a829-698854d99d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d3a829: serviceaccount "payload-processing-e2e-ipp-a-d3a829" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-d3a829-698854d99d Created pod: payload-processing-e2e-ipp-a-d3a829-698854d99d-8f69p 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-d3a829 Scaled up replica set payload-processing-e2e-ipp-a-d3a829-698854d99d from 0 to 1 35m Normal Created pod/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-xlcmh Container created 35m Normal Created pod/payload-processing-e2e-ipp-a-d3a829-698854d99d-8f69p Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-xlcmh Add eth0 [10.132.0.189/23] from ovn-kubernetes 35m Normal AddedInterface pod/payload-processing-e2e-ipp-a-d3a829-698854d99d-8f69p Add eth0 [10.132.0.190/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-ipp-a-d3a829-698854d99d-8f69p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-xlcmh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-xlcmh Container started 35m Normal Started pod/payload-processing-e2e-ipp-a-d3a829-698854d99d-8f69p Container started 34m Normal Started pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Container started 34m Normal Created pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Container created 34m Normal Pulled pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning ProbeError pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Startup probe error: Get "http://10.133.1.10:15021/healthz/ready": dial tcp 10.133.1.10:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Startup probe failed: Get "http://10.133.1.10:15021/healthz/ready": dial tcp 10.133.1.10:15021: connect: connection refused 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e50e63-openshift-default Scaled up replica set e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc Created pod: e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn 34m Normal AddedLabel gateway/e2e-shared-b-w1-e50e63 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-e50e63 34m Normal AddedInterface pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn Add eth0 [10.133.1.19/23] from ovn-kubernetes 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-f1f206-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-f1f206-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Created pod/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx Container created 33m Normal Pulled pod/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl Container started 33m Normal Pulled pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl Container created 33m Normal Started pod/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx Container started 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-c7c37b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-c7c37b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5 Created pod: payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-e50e63 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5 from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd Created pod: payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 33m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd Error creating: pods "payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-e50e63: serviceaccount "payload-processing-e2e-shared-b-w1-e50e63" not found 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-e50e63 Scaled up replica set payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd from 0 to 1 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-e50e63: serviceaccount "payload-processing-e2e-shared-b-w1-e50e63" not found 33m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn Add eth0 [10.134.0.188/23] from ovn-kubernetes 33m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Warning Unhealthy pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl Startup probe failed: Get "http://10.133.1.13:15021/healthz/ready": dial tcp 10.133.1.13:15021: connect: connection refused 33m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 Add eth0 [10.134.0.189/23] from ovn-kubernetes 33m Normal Pulled pod/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 Container created 33m Normal Started pod/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 Container started 33m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-6e91df-openshift-default Scaled up replica set e2e-shared-b-w2-6e91df-openshift-default-8689d4c967 from 1 to 2 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-6e91df-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Warning Unhealthy pod/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx Startup probe failed: Get "http://10.133.1.16:15021/healthz/ready": dial tcp 10.133.1.16:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl Startup probe error: Get "http://10.133.1.13:15021/healthz/ready": dial tcp 10.133.1.13:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx Startup probe error: Get "http://10.133.1.16:15021/healthz/ready": dial tcp 10.133.1.16:15021: connect: connection refused... 33m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn Container created 33m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967 Created pod: e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 33m Normal AddedInterface pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 Add eth0 [10.134.0.190/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn Container started 33m Normal Started pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 Container started 33m Normal Pulled pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 Container created 33m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-a59ecb-openshift-default Scaled up replica set e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 from 2 to 3 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-a59ecb-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 Created pod: e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp 33m Normal Created pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp Container created 33m Normal Started pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp Container started 33m Normal Pulled pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp Add eth0 [10.134.0.191/23] from ovn-kubernetes 33m Warning Unhealthy pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 Startup probe failed: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 Startup probe error: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp Startup probe error: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp Startup probe failed: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-d3a829-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-d3a829-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/payload-pre-processing-e2e-ipp-a-d3a829-6fcfc75547-xlcmh Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-ipp-a-d3a829-698854d99d-8f69p Stopping container payload-processing 33m Normal ScalingReplicaSet deployment/e2e-worker-w6-39529e-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-39529e-openshift-default-cb57f9db7 from 2 to 1 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-39529e-openshift-default New size: 1; reason: All metrics below target 33m Normal SuccessfulDelete replicaset/e2e-worker-w6-39529e-openshift-default-cb57f9db7 (combined from similar events): Deleted pod: e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 33m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-mqmb4 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-d3a829-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-c7c37b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-d3a829-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-c7c37b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal AddedLabel gateway/e2e-ipp-a-f0e8b9 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f0e8b9 33m Normal SuccessfulCreate replicaset/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f Created pod: e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z 33m Normal ScalingReplicaSet deployment/e2e-ipp-a-f0e8b9-openshift-default Scaled up replica set e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f from 0 to 1 33m Normal AddedInterface pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z Add eth0 [10.133.1.22/23] from ovn-kubernetes 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79 Error creating: pods "payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d3a829: serviceaccount "payload-processing-e2e-ipp-a-d3a829" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79 Created pod: payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 33m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76 Error creating: pods "payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d3a829: serviceaccount "payload-processing-e2e-ipp-a-d3a829" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76 Created pod: payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-d3a829 Scaled up replica set payload-processing-e2e-ipp-a-d3a829-77fcbf8b76 from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-d3a829 Scaled up replica set payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79 from 0 to 1 32m Normal ScalingReplicaSet deployment/e2e-worker-w4-46b13b-openshift-default Scaled down replica set e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c from 2 to 1 32m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Stopping container istio-proxy 32m Normal AddedInterface pod/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 Add eth0 [10.133.1.23/23] from ovn-kubernetes 32m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f Deleted pod: e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-da78c4-openshift-default New size: 1; reason: All metrics below target 32m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 32m Normal Pulling pod/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-46b13b-openshift-default New size: 1; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c Deleted pod: e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 Add eth0 [10.132.0.201/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/e2e-worker-w2-da78c4-openshift-default Scaled down replica set e2e-worker-w2-da78c4-openshift-default-84dbb4c47f from 2 to 1 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.316s (1.316s including waiting). Image size: 166612641 bytes. 32m Normal Started pod/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 Container started 32m Normal Created pod/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 Container created 32m Normal Created pod/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 Container created 32m Normal Started pod/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 Container started 32m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-r4bft 32m Normal Pulled pod/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.44s (1.44s including waiting). Image size: 166612641 bytes. 32m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-r4bft Stopping container discovery 32m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-g9h4w Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-9e3aa0-openshift-default Scaled up replica set e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f from 1 to 2 32m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-9e3aa0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-9sr84 Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f Created pod: e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs 32m Normal Started pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs Container started 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f0e8b9 Scaled up replica set payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f0e8b9 Scaled up replica set payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48 from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48 Created pod: payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-f2lwf 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48 Error creating: pods "payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f0e8b9: serviceaccount "payload-processing-e2e-ipp-a-f0e8b9" not found 32m Normal Created pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs Container created 32m Normal AddedInterface pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs Add eth0 [10.134.0.192/23] from ovn-kubernetes 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587 Error creating: pods "payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f0e8b9: serviceaccount "payload-processing-e2e-ipp-a-f0e8b9" not found 32m Normal Pulled pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587 Created pod: payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-gw4r5 32m Normal Started pod/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-gw4r5 Container started 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-gw4r5 Add eth0 [10.134.0.193/23] from ovn-kubernetes 32m Normal Started pod/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-f2lwf Container started 32m Normal Pulled pod/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-f2lwf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-f2lwf Add eth0 [10.134.0.194/23] from ovn-kubernetes 32m Normal Created pod/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-f2lwf Container created 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-gw4r5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-gw4r5 Container created 32m Warning ProbeError pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs Startup probe error: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs Startup probe failed: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused 32m Normal ScalingReplicaSet deployment/e2e-ipp-b-460b0b-openshift-default Scaled up replica set e2e-ipp-b-460b0b-openshift-default-596d6cdf7 from 0 to 1 32m Normal AddedLabel gateway/e2e-ipp-b-460b0b Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-460b0b 32m Normal SuccessfulCreate replicaset/e2e-ipp-b-460b0b-openshift-default-596d6cdf7 Created pod: e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js 32m Normal AddedInterface pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Add eth0 [10.132.0.202/23] from ovn-kubernetes 32m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-6cf72 32m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 1; reason: All metrics below target 32m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 2 to 1 32m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-6cf72 Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-f1f206-openshift-default Scaled up replica set e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9 from 1 to 2 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9 Error creating: pods "payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-460b0b: serviceaccount "payload-processing-e2e-ipp-b-460b0b" not found 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-460b0b Scaled up replica set payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9 from 0 to 1 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-f1f206-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9 Created pod: e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-460b0b-86df895954 Error creating: pods "payload-processing-e2e-ipp-b-460b0b-86df895954-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-460b0b: serviceaccount "payload-processing-e2e-ipp-b-460b0b" not found 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-460b0b Scaled up replica set payload-processing-e2e-ipp-b-460b0b-86df895954 from 0 to 1 32m Normal Pulled pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm Container created 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-460b0b-86df895954 Created pod: payload-processing-e2e-ipp-b-460b0b-86df895954-nvbqg 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9 Created pod: payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-ct6xg 32m Normal AddedInterface pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm Add eth0 [10.134.0.195/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-ct6xg Container started 32m Normal Created pod/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-ct6xg Container created 32m Normal Started pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm Container started 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-ct6xg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-ipp-b-460b0b-86df895954-nvbqg Add eth0 [10.134.0.197/23] from ovn-kubernetes 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-ct6xg Add eth0 [10.134.0.196/23] from ovn-kubernetes 32m Normal Pulled pod/payload-processing-e2e-ipp-b-460b0b-86df895954-nvbqg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-processing-e2e-ipp-b-460b0b-86df895954-nvbqg Container created 32m Normal Started pod/payload-processing-e2e-ipp-b-460b0b-86df895954-nvbqg Container started 32m Warning ProbeError pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm Startup probe error: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm Startup probe failed: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused 32m Normal AddedLabel gateway/e2e-ipp-a-33c209 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-33c209 32m Normal AddedInterface pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Add eth0 [10.134.0.199/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn Container created 32m Normal Started pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn Container started 32m Normal SuccessfulCreate replicaset/e2e-ipp-a-33c209-openshift-default-8dc5b9dc Created pod: e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m 32m Normal ScalingReplicaSet deployment/e2e-ipp-a-33c209-openshift-default Scaled up replica set e2e-ipp-a-33c209-openshift-default-8dc5b9dc from 0 to 1 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-396d70-openshift-default Scaled up replica set e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65 from 1 to 2 32m Normal Started pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc Container started 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65 Created pod: e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc 32m Normal Pulled pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-396d70-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal AddedInterface pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc Add eth0 [10.134.0.200/23] from ovn-kubernetes 32m Normal Created pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc Container created 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-75443e-openshift-default Scaled up replica set e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc from 1 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-75443e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-33c209 Scaled up replica set payload-processing-e2e-ipp-a-33c209-6f4597cc68 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-33c209 Scaled up replica set payload-pre-processing-e2e-ipp-a-33c209-bffd664d7 from 0 to 1 32m Normal Pulled pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Add eth0 [10.134.0.201/23] from ovn-kubernetes 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7 Error creating: pods "payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-33c209: serviceaccount "payload-processing-e2e-ipp-a-33c209" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7 Created pod: payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-kcz62 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-33c209-6f4597cc68 Error creating: pods "payload-processing-e2e-ipp-a-33c209-6f4597cc68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-33c209: serviceaccount "payload-processing-e2e-ipp-a-33c209" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-33c209-6f4597cc68 Created pod: payload-processing-e2e-ipp-a-33c209-6f4597cc68-xwdkx 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc Created pod: e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m 32m Normal Created pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Container created 32m Normal Started pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Container started 32m Normal Pulled pod/payload-processing-e2e-ipp-a-33c209-6f4597cc68-xwdkx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-0efe43-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-kcz62 Add eth0 [10.132.0.208/23] from ovn-kubernetes 32m Normal Created pod/payload-processing-e2e-ipp-a-33c209-6f4597cc68-xwdkx Container created 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-kcz62 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-kcz62 Container started 32m Warning Unhealthy pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn Startup probe failed: Get "http://10.133.1.19:15021/healthz/ready": dial tcp 10.133.1.19:15021: connect: connection refused 32m Normal Started pod/payload-processing-e2e-ipp-a-33c209-6f4597cc68-xwdkx Container started 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0efe43-openshift-default Scaled up replica set e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99 from 1 to 2 32m Normal AddedInterface pod/payload-processing-e2e-ipp-a-33c209-6f4597cc68-xwdkx Add eth0 [10.132.0.209/23] from ovn-kubernetes 32m Warning ProbeError pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn Startup probe error: Get "http://10.133.1.19:15021/healthz/ready": dial tcp 10.133.1.19:15021: connect: connection refused... 32m Normal Created pod/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-kcz62 Container created 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99 Created pod: e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js 32m Warning ProbeError pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc Startup probe error: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused... 32m Normal Started pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js Container started 32m Warning Unhealthy pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc Startup probe failed: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused 32m Normal Created pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js Container created 32m Normal Pulled pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js Add eth0 [10.133.1.25/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js Startup probe error: Get "http://10.133.1.25:15021/healthz/ready": dial tcp 10.133.1.25:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js Startup probe failed: Get "http://10.133.1.25:15021/healthz/ready": dial tcp 10.133.1.25:15021: connect: connection refused 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-e50e63-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-e50e63-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-e50e63-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-e50e63-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal AddedLabel gateway/e2e-ipp-a-6176c3 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-6176c3 31m Normal SuccessfulCreate replicaset/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f Created pod: e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j 31m Normal ScalingReplicaSet deployment/e2e-ipp-a-6176c3-openshift-default Scaled up replica set e2e-ipp-a-6176c3-openshift-default-64ddffcc7f from 0 to 1 31m Normal AddedInterface pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Add eth0 [10.133.1.27/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f Created pod: e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 31m Normal AddedLabel gateway/e2e-ipp-a-a1854b Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-a1854b 31m Normal ScalingReplicaSet deployment/e2e-ipp-a-a1854b-openshift-default Scaled up replica set e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f from 0 to 1 31m Normal AddedInterface pod/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 Add eth0 [10.132.0.215/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c Created pod: payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-dr679 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c Error creating: pods "payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a1854b: serviceaccount "payload-processing-e2e-ipp-a-a1854b" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a1854b-6bd77d5846 Created pod: payload-processing-e2e-ipp-a-a1854b-6bd77d5846-984lk 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a1854b-6bd77d5846 Error creating: pods "payload-processing-e2e-ipp-a-a1854b-6bd77d5846-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a1854b: serviceaccount "payload-processing-e2e-ipp-a-a1854b" not found 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a1854b Scaled up replica set payload-processing-e2e-ipp-a-a1854b-6bd77d5846 from 0 to 1 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a1854b Scaled up replica set payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c from 0 to 1 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-dr679 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-dr679 Add eth0 [10.134.0.202/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a1854b-6bd77d5846-984lk Add eth0 [10.132.0.216/23] from ovn-kubernetes 31m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f0e8b9-74b8f4b587-gw4r5 Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-ipp-a-f0e8b9-7759fdbc48-f2lwf Stopping container payload-processing 31m Normal Created pod/payload-processing-e2e-ipp-a-a1854b-6bd77d5846-984lk Container created 31m Normal Started pod/payload-processing-e2e-ipp-a-a1854b-6bd77d5846-984lk Container started 31m Normal Pulled pod/payload-processing-e2e-ipp-a-a1854b-6bd77d5846-984lk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-dr679 Container created 31m Normal Started pod/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-dr679 Container started 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-6176c3 Scaled up replica set payload-processing-e2e-ipp-a-6176c3-749f86c99b from 0 to 1 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-6176c3-749f86c99b Error creating: pods "payload-processing-e2e-ipp-a-6176c3-749f86c99b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-6176c3: serviceaccount "payload-processing-e2e-ipp-a-6176c3" not found 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59 Error creating: pods "payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-6176c3: serviceaccount "payload-processing-e2e-ipp-a-6176c3" not found 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59 Created pod: payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-qkgkv 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-6176c3-749f86c99b Created pod: payload-processing-e2e-ipp-a-6176c3-749f86c99b-wfgdk 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-6176c3 Scaled up replica set payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59 from 0 to 1 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-qkgkv Add eth0 [10.134.0.203/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-processing-e2e-ipp-a-6176c3-749f86c99b-wfgdk Add eth0 [10.132.0.217/23] from ovn-kubernetes 31m Normal Pulled pod/payload-processing-e2e-ipp-a-6176c3-749f86c99b-wfgdk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-processing-e2e-ipp-a-6176c3-749f86c99b-wfgdk Container created 31m Normal Started pod/payload-processing-e2e-ipp-a-6176c3-749f86c99b-wfgdk Container started 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-qkgkv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-qkgkv Container created 31m Normal Started pod/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-qkgkv Container started 30m Normal Created pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z Container created 30m Normal Pulled pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Pulled pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z Container started 30m Normal Started pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Container started 30m Normal Created pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Container created 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-75443e-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-75443e-openshift-default Scaled up replica set e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc from 2 to 3 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc Created pod: e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz 30m Normal AddedInterface pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz Add eth0 [10.132.0.218/23] from ovn-kubernetes 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0efe43-openshift-default Scaled up replica set e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99 from 2 to 3 30m Normal Pulled pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99 Created pod: e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q 30m Normal Started pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz Container started 30m Normal Created pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz Container created 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-0efe43-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 30m Normal AddedInterface pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q Add eth0 [10.134.0.205/23] from ovn-kubernetes 30m Normal Started pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q Container started 30m Normal Pulled pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q Container created 30m Warning ProbeError pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z Startup probe error: Get "http://10.133.1.22:15021/healthz/ready": dial tcp 10.133.1.22:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z Startup probe failed: Get "http://10.133.1.22:15021/healthz/ready": dial tcp 10.133.1.22:15021: connect: connection refused 30m Warning ProbeError pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Startup probe error: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Startup probe failed: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused 30m Normal Killing pod/payload-pre-processing-e2e-ipp-b-460b0b-77fcc4cc9-ct6xg Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-ipp-b-460b0b-86df895954-nvbqg Stopping container payload-processing 30m Warning Unhealthy pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz Startup probe failed: Get "http://10.132.0.218:15021/healthz/ready": dial tcp 10.132.0.218:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz Startup probe error: Get "http://10.132.0.218:15021/healthz/ready": dial tcp 10.132.0.218:15021: connect: connection refused... 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a1854b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning Unhealthy pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q Startup probe failed: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a1854b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning ProbeError pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q Startup probe error: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused... 30m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-68bcg 30m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 30m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-68bcg Container started 30m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-68bcg Container created 30m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-68bcg Add eth0 [10.134.0.207/23] from ovn-kubernetes 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f0e8b9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-68bcg Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f0e8b9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-460b0b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-460b0b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-68bcg Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-68bcg Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal SuccessfulCreate replicaset/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7 Created pod: e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 30m Normal ScalingReplicaSet deployment/e2e-ipp-a-2927fd-openshift-default Scaled up replica set e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7 from 0 to 1 30m Normal AddedLabel gateway/e2e-ipp-a-2927fd Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-2927fd 30m Normal AddedInterface pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Add eth0 [10.133.1.29/23] from ovn-kubernetes 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f0e8b9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f0e8b9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59 Created pod: e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-c7c37b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-c7c37b-openshift-default Scaled up replica set e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59 from 1 to 2 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-460b0b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-460b0b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal Started pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 Container started 30m Normal Created pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 Container created 30m Normal AddedInterface pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 Add eth0 [10.132.0.219/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-2927fd Scaled up replica set payload-pre-processing-e2e-ipp-a-2927fd-55db459cf from 0 to 1 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-2927fd Scaled up replica set payload-processing-e2e-ipp-a-2927fd-844ddcc4bd from 0 to 1 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd Error creating: pods "payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2927fd: serviceaccount "payload-processing-e2e-ipp-a-2927fd" not found 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf Error creating: pods "payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2927fd: serviceaccount "payload-processing-e2e-ipp-a-2927fd" not found 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf Created pod: payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-lqzdk 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd Created pod: payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-wbrs7 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-lqzdk Add eth0 [10.132.0.220/23] from ovn-kubernetes 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-lqzdk Container started 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-lqzdk Container created 30m Normal Started pod/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-wbrs7 Container started 30m Normal Created pod/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-wbrs7 Container created 30m Normal Pulled pod/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-wbrs7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-lqzdk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-wbrs7 Add eth0 [10.132.0.221/23] from ovn-kubernetes 30m Warning Unhealthy pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused... 30m Normal Killing pod/payload-processing-e2e-ipp-a-33c209-6f4597cc68-xwdkx Stopping container payload-processing 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98 Error creating: pods "payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f0e8b9: serviceaccount "payload-processing-e2e-ipp-a-f0e8b9" not found 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98 Created pod: payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-7942f 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f0e8b9 Scaled up replica set payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98 from 0 to 1 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9 Error creating: pods "payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f0e8b9: serviceaccount "payload-processing-e2e-ipp-a-f0e8b9" not found 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9 Created pod: payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-jdqzn 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f0e8b9 Scaled up replica set payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9 from 0 to 1 30m Normal Killing pod/payload-pre-processing-e2e-ipp-a-33c209-bffd664d7-kcz62 Stopping container payload-pre-processing 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-jdqzn Add eth0 [10.133.1.30/23] from ovn-kubernetes 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-jdqzn Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-jdqzn Container started 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-jdqzn Container created 30m Normal Started pod/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-7942f Container started 30m Normal Created pod/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-7942f Container created 30m Normal Pulled pod/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-7942f Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-7942f Add eth0 [10.133.1.31/23] from ovn-kubernetes 30m Normal Created pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Container created 30m Normal Pulled pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Warning ProbeError pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Readiness probe error: Get "http://10.134.0.199:15021/healthz/ready": read tcp 10.134.0.2:41146->10.134.0.199:15021: read: connection reset by peer... 30m Normal Started pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Container started 30m Warning Unhealthy pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Readiness probe failed: Get "http://10.134.0.199:15021/healthz/ready": read tcp 10.134.0.2:41146->10.134.0.199:15021: read: connection reset by peer 30m Warning Unhealthy pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Startup probe failed: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused 30m Warning ProbeError pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Startup probe error: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused... 30m Normal AddedLabel gateway/e2e-ipp-b-3b95dd Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-3b95dd 30m Normal ScalingReplicaSet deployment/e2e-ipp-b-3b95dd-openshift-default Scaled up replica set e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9 from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9 Created pod: e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr 30m Normal AddedInterface pod/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr Add eth0 [10.133.1.33/23] from ovn-kubernetes 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-33c209-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-33c209-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c Error creating: pods "payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3b95dd: serviceaccount "payload-processing-e2e-ipp-b-3b95dd" not found 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-3b95dd Scaled up replica set payload-processing-e2e-ipp-b-3b95dd-5ccb7d956 from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956 Created pod: payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-4mkxk 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-3b95dd Scaled up replica set payload-pre-processing-e2e-ipp-b-3b95dd-858549854c from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c Created pod: payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-z49d4 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956 Error creating: pods "payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3b95dd: serviceaccount "payload-processing-e2e-ipp-b-3b95dd" not found 29m Normal Created pod/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-z49d4 Container created 29m Normal AddedInterface pod/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-4mkxk Add eth0 [10.132.0.224/23] from ovn-kubernetes 29m Normal Started pod/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-z49d4 Container started 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-z49d4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-z49d4 Add eth0 [10.132.0.223/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-4mkxk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-4mkxk Container created 29m Normal Started pod/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-4mkxk Container started 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-460b0b-55d85f88fb Error creating: pods "payload-processing-e2e-ipp-b-460b0b-55d85f88fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-460b0b: serviceaccount "payload-processing-e2e-ipp-b-460b0b" not found 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-460b0b Scaled up replica set payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955 from 0 to 1 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955 Error creating: pods "payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-460b0b: serviceaccount "payload-processing-e2e-ipp-b-460b0b" not found 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-460b0b Scaled up replica set payload-processing-e2e-ipp-b-460b0b-55d85f88fb from 0 to 1 29m Normal AddedInterface pod/payload-processing-e2e-ipp-b-460b0b-55d85f88fb-7rkhm Add eth0 [10.132.0.225/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-ipp-b-460b0b-55d85f88fb-7rkhm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-processing-e2e-ipp-b-460b0b-55d85f88fb-7rkhm Container started 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955 Created pod: payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-460b0b-55d85f88fb Created pod: payload-processing-e2e-ipp-b-460b0b-55d85f88fb-7rkhm 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n Add eth0 [10.133.1.34/23] from ovn-kubernetes 29m Normal Created pod/payload-processing-e2e-ipp-b-460b0b-55d85f88fb-7rkhm Container created 29m Normal Started pod/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n Container started 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n Container created 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-33c209-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-33c209-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc Created pod: e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-e50e63-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e50e63-openshift-default Scaled up replica set e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc from 1 to 2 29m Normal Pulled pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Add eth0 [10.132.0.226/23] from ovn-kubernetes 29m Normal Started pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Container started 29m Normal Created pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Container created 29m Warning ProbeError pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Startup probe error: Get "http://10.132.0.226:15021/healthz/ready": dial tcp 10.132.0.226:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Startup probe failed: Get "http://10.132.0.226:15021/healthz/ready": dial tcp 10.132.0.226:15021: connect: connection refused 29m Normal Started pod/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 Container started 29m Normal Created pod/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 Container created 29m Normal Pulled pod/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Pulled pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Warning ProbeError pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Readiness probe error: Get "http://10.133.1.27:15021/healthz/ready": dial tcp 10.133.1.27:15021: connect: connection refused... 29m Normal Started pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Container started 29m Normal Created pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Container created 29m Warning Unhealthy pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Readiness probe failed: Get "http://10.133.1.27:15021/healthz/ready": dial tcp 10.133.1.27:15021: connect: connection refused 29m Warning ProbeError pod/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 Startup probe error: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 Startup probe failed: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused 29m Warning ProbeError pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Startup probe error: Get "http://10.133.1.27:15021/healthz/ready": dial tcp 10.133.1.27:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Startup probe failed: Get "http://10.133.1.27:15021/healthz/ready": dial tcp 10.133.1.27:15021: connect: connection refused 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-33c209 Scaled up replica set payload-pre-processing-e2e-ipp-a-33c209-5b7b659959 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-33c209 Scaled up replica set payload-processing-e2e-ipp-a-33c209-59b7c984c4 from 0 to 1 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-33c209-59b7c984c4 Error creating: pods "payload-processing-e2e-ipp-a-33c209-59b7c984c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-33c209: serviceaccount "payload-processing-e2e-ipp-a-33c209" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-33c209-59b7c984c4 Created pod: payload-processing-e2e-ipp-a-33c209-59b7c984c4-9wlm7 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-97sdt Add eth0 [10.132.0.227/23] from ovn-kubernetes 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959 Created pod: payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-97sdt 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959 Error creating: pods "payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-33c209: serviceaccount "payload-processing-e2e-ipp-a-33c209" not found 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-97sdt Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-97sdt Container created 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-97sdt Container started 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-33c209-59b7c984c4-9wlm7 Add eth0 [10.133.1.35/23] from ovn-kubernetes 29m Normal Started pod/payload-processing-e2e-ipp-a-33c209-59b7c984c4-9wlm7 Container started 29m Normal Created pod/payload-processing-e2e-ipp-a-33c209-59b7c984c4-9wlm7 Container created 29m Normal Pulled pod/payload-processing-e2e-ipp-a-33c209-59b7c984c4-9wlm7 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-6176c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-6176c3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a1854b-9d64cdc8c-dr679 Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-ipp-a-a1854b-6bd77d5846-984lk Stopping container payload-processing 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-6176c3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-6176c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-6176c3-f464c4b59-qkgkv Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-ipp-a-6176c3-749f86c99b-wfgdk Stopping container payload-processing 28m Normal SuccessfulCreate replicaset/e2e-ipp-b-d05e21-openshift-default-6c99db878c Created pod: e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp 28m Normal ScalingReplicaSet deployment/e2e-ipp-b-d05e21-openshift-default Scaled up replica set e2e-ipp-b-d05e21-openshift-default-6c99db878c from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-b-d05e21 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-d05e21 28m Normal AddedInterface pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp Add eth0 [10.132.0.228/23] from ovn-kubernetes 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a1854b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a1854b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c Created pod: payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-hmlcp 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d Error creating: pods "payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d05e21: serviceaccount "payload-processing-e2e-ipp-b-d05e21" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-d05e21 Scaled up replica set payload-processing-e2e-ipp-b-d05e21-6796c5fc4d from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d Created pod: payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-xzzzk 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c Error creating: pods "payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d05e21: serviceaccount "payload-processing-e2e-ipp-b-d05e21" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-d05e21 Scaled up replica set payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c from 0 to 1 28m Normal Pulled pod/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-xzzzk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-xzzzk Add eth0 [10.134.0.213/23] from ovn-kubernetes 28m Normal Created pod/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-xzzzk Container created 28m Normal Started pod/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-xzzzk Container started 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-hmlcp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-hmlcp Container created 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-hmlcp Add eth0 [10.132.0.229/23] from ovn-kubernetes 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-hmlcp Container started 28m Warning Unhealthy pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Readiness probe failed: Get "http://10.133.1.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Warning ProbeError pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Readiness probe error: Get "http://10.133.1.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-d05e21-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-d05e21-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-d3a829-64f6b74c79-6wf64 Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-ipp-a-d3a829-77fcbf8b76-8q8h9 Stopping container payload-processing 28m Normal Pulled pod/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr Container started 28m Normal Created pod/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr Container created 28m Normal Pulled pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Container started 28m Normal Created pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Container created 28m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m Stopping container payload-processing 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-3b95dd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-3b95dd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning Unhealthy pod/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr Startup probe failed: Get "http://10.133.1.33:15021/healthz/ready": dial tcp 10.133.1.33:15021: connect: connection refused 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-2927fd-55db459cf-lqzdk Stopping container payload-pre-processing 28m Warning ProbeError pod/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr Startup probe error: Get "http://10.133.1.33:15021/healthz/ready": dial tcp 10.133.1.33:15021: connect: connection refused... 28m Normal Killing pod/payload-processing-e2e-ipp-a-2927fd-844ddcc4bd-wbrs7 Stopping container payload-processing 28m Warning Unhealthy pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Startup probe failed: Get "http://10.133.1.29:15021/healthz/ready": dial tcp 10.133.1.29:15021: connect: connection refused 28m Warning ProbeError pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Startup probe error: Get "http://10.133.1.29:15021/healthz/ready": dial tcp 10.133.1.29:15021: connect: connection refused... 28m Normal Killing pod/e2e-ipp-a-d3a829-openshift-default-74d65df7df-q2hjx Stopping container istio-proxy 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-2927fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-2927fd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575 Created pod: payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-prc47 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a1854b-58595bff5b Error creating: pods "payload-processing-e2e-ipp-a-a1854b-58595bff5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a1854b: serviceaccount "payload-processing-e2e-ipp-a-a1854b" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a1854b-58595bff5b Created pod: payload-processing-e2e-ipp-a-a1854b-58595bff5b-vvvvd 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575 Error creating: pods "payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a1854b: serviceaccount "payload-processing-e2e-ipp-a-a1854b" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a1854b Scaled up replica set payload-processing-e2e-ipp-a-a1854b-58595bff5b from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a1854b Scaled up replica set payload-pre-processing-e2e-ipp-a-a1854b-64bc599575 from 0 to 1 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-prc47 Add eth0 [10.133.1.36/23] from ovn-kubernetes 28m Normal Created pod/payload-processing-e2e-ipp-a-a1854b-58595bff5b-vvvvd Container created 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a1854b-58595bff5b-vvvvd Add eth0 [10.133.1.37/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-a-a1854b-58595bff5b-vvvvd Container started 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-prc47 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-prc47 Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-prc47 Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-a-a1854b-58595bff5b-vvvvd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-c7c37b-6b9c5c5999-kdfrr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-6176c3 Scaled up replica set payload-processing-e2e-ipp-a-6176c3-65cd57864b from 0 to 1 28m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-c7c37b-67bb95b964-cqz6m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf Created pod: payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-h2hqx 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf Error creating: pods "payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-6176c3: serviceaccount "payload-processing-e2e-ipp-a-6176c3" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-6176c3-65cd57864b Created pod: payload-processing-e2e-ipp-a-6176c3-65cd57864b-tqgc2 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-6176c3 Scaled up replica set payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-6176c3-65cd57864b Error creating: pods "payload-processing-e2e-ipp-a-6176c3-65cd57864b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-6176c3: serviceaccount "payload-processing-e2e-ipp-a-6176c3" not found 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-h2hqx Container started 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-h2hqx Container created 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-h2hqx Add eth0 [10.133.1.38/23] from ovn-kubernetes 28m Normal Created pod/payload-processing-e2e-ipp-a-6176c3-65cd57864b-tqgc2 Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-h2hqx Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Normal Pulled pod/payload-processing-e2e-ipp-a-6176c3-65cd57864b-tqgc2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-6176c3-65cd57864b-tqgc2 Add eth0 [10.133.1.39/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-a-6176c3-65cd57864b-tqgc2 Container started 27m Normal Killing pod/payload-pre-processing-e2e-ipp-b-3b95dd-858549854c-z49d4 Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-ipp-b-3b95dd-5ccb7d956-4mkxk Stopping container payload-processing 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-4be240-openshift-default Scaled up replica set e2e-ipp-b-4be240-openshift-default-5b576754f5 from 0 to 1 27m Normal AddedLabel gateway/e2e-ipp-b-4be240 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-4be240 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-4be240-openshift-default-5b576754f5 Created pod: e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 27m Normal AddedInterface pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Add eth0 [10.133.1.40/23] from ovn-kubernetes 27m Normal Killing pod/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n Stopping container payload-pre-processing 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-90b406-openshift-default-68c56c8459 Created pod: e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-90b406-openshift-default Scaled up replica set e2e-ipp-b-90b406-openshift-default-68c56c8459 from 0 to 1 27m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-b-460b0b-6b4dbb5955-std4n Readiness probe failed: dial tcp 10.133.1.34:9004: connect: connection refused 27m Normal Killing pod/payload-processing-e2e-ipp-b-460b0b-55d85f88fb-7rkhm Stopping container payload-processing 27m Normal AddedLabel gateway/e2e-ipp-b-90b406 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-90b406 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-3b95dd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-3b95dd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Killing pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-zgfl2 Stopping container istio-proxy 27m Normal AddedInterface pod/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 Add eth0 [10.132.0.232/23] from ovn-kubernetes 27m Normal Killing pod/e2e-shared-a-w1-c7c37b-openshift-default-6795b6cf59-lrnfl Stopping container istio-proxy 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c Error creating: pods "payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4be240: serviceaccount "payload-processing-e2e-ipp-b-4be240" not found 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb Created pod: payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-n8p5d 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c Created pod: payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-7z9qv 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-4be240 Scaled up replica set payload-processing-e2e-ipp-b-4be240-58b8ddbdcb from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-4be240 Scaled up replica set payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb Error creating: pods "payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4be240: serviceaccount "payload-processing-e2e-ipp-b-4be240" not found 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-7z9qv Add eth0 [10.132.0.233/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-n8p5d Add eth0 [10.132.0.234/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-7z9qv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-7z9qv Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-7z9qv Container started 27m Normal Pulled pod/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-n8p5d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-n8p5d Container created 27m Normal Started pod/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-n8p5d Container started 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-2927fd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-2927fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-90b406-548d97d98d Created pod: payload-processing-e2e-ipp-b-90b406-548d97d98d-tjsf4 27m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f Deleted pod: e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-9e3aa0-openshift-default New size: 1; reason: All metrics below target 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-90b406-548d97d98d Error creating: pods "payload-processing-e2e-ipp-b-90b406-548d97d98d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-90b406: serviceaccount "payload-processing-e2e-ipp-b-90b406" not found 27m Normal Killing pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Stopping container istio-proxy 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7 Created pod: payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-4wb24 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7 Error creating: pods "payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-90b406: serviceaccount "payload-processing-e2e-ipp-b-90b406" not found 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-9e3aa0-openshift-default Scaled down replica set e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f from 2 to 1 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-90b406 Scaled up replica set payload-processing-e2e-ipp-b-90b406-548d97d98d from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-90b406 Scaled up replica set payload-pre-processing-e2e-ipp-b-90b406-98b94bb7 from 0 to 1 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-4wb24 Container started 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-90b406-548d97d98d-tjsf4 Add eth0 [10.132.0.236/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-4wb24 Add eth0 [10.132.0.235/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-b-90b406-548d97d98d-tjsf4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-processing-e2e-ipp-b-90b406-548d97d98d-tjsf4 Container created 27m Normal Started pod/payload-processing-e2e-ipp-b-90b406-548d97d98d-tjsf4 Container started 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-4wb24 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-4wb24 Container created 27m Warning Unhealthy pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-4p5h4 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Stopping container istio-proxy 27m Warning Unhealthy pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-ipp-b-460b0b-openshift-default-596d6cdf7-x67js Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-a59ecb-openshift-default New size: 2; reason: All metrics below target 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-a59ecb-openshift-default Scaled down replica set e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 from 3 to 2 27m Normal SuccessfulDelete replicaset/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 Deleted pod: e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx 27m Normal Killing pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-gqvzx Stopping container istio-proxy 27m Normal Killing pod/payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn Stopping container payload-pre-processing 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9 Error creating: pods "payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2927fd: serviceaccount "payload-processing-e2e-ipp-a-2927fd" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-2927fd Scaled up replica set payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9 from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-2927fd-58947b47f4 Error creating: pods "payload-processing-e2e-ipp-a-2927fd-58947b47f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2927fd: serviceaccount "payload-processing-e2e-ipp-a-2927fd" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-2927fd Scaled up replica set payload-processing-e2e-ipp-a-2927fd-58947b47f4 from 0 to 1 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-p2hhp Container created 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-2927fd-58947b47f4 Created pod: payload-processing-e2e-ipp-a-2927fd-58947b47f4-wh6th 27m Normal Created pod/payload-processing-e2e-ipp-a-2927fd-58947b47f4-wh6th Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-p2hhp Container started 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-p2hhp Add eth0 [10.132.0.239/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-a-2927fd-58947b47f4-wh6th Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9 Created pod: payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-p2hhp 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-2927fd-58947b47f4-wh6th Add eth0 [10.132.0.240/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-p2hhp Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Started pod/payload-processing-e2e-ipp-a-2927fd-58947b47f4-wh6th Container started 27m Normal Killing pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-228454-openshift-default Scaled down replica set e2e-shared-b-w3-228454-openshift-default-6476dbf645 from 2 to 1 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-228454-openshift-default New size: 1; reason: All metrics below target 27m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-228454-openshift-default-6476dbf645 Deleted pod: e2e-shared-b-w3-228454-openshift-default-6476dbf645-p4t2s 27m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-acefbb-5c87485f47-l4jfk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-acefbb-569588ff76-rxwvd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-bcc710-openshift-default-645587584f Created pod: e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t 27m Normal AddedLabel gateway/e2e-ipp-b-bcc710 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-bcc710 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-bcc710-openshift-default Scaled up replica set e2e-ipp-b-bcc710-openshift-default-645587584f from 0 to 1 27m Normal AddedInterface pod/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t Add eth0 [10.133.1.42/23] from ovn-kubernetes 26m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-e50e63-585ff6bcbd-jlsv6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-e50e63-7674c8c9f5-nggnn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-bcc710 Scaled up replica set payload-pre-processing-e2e-ipp-b-bcc710-76656769cf from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-bcc710 Scaled up replica set payload-processing-e2e-ipp-b-bcc710-7478845cd from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf Created pod: payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-dmk7s 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf Error creating: pods "payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bcc710: serviceaccount "payload-processing-e2e-ipp-b-bcc710" not found 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-dmk7s Add eth0 [10.132.0.242/23] from ovn-kubernetes 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-bcc710-7478845cd Error creating: pods "payload-processing-e2e-ipp-b-bcc710-7478845cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bcc710: serviceaccount "payload-processing-e2e-ipp-b-bcc710" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-bcc710-7478845cd Created pod: payload-processing-e2e-ipp-b-bcc710-7478845cd-x7lwp 26m Normal Started pod/payload-processing-e2e-ipp-b-bcc710-7478845cd-x7lwp Container started 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-bcc710-7478845cd-x7lwp Add eth0 [10.132.0.241/23] from ovn-kubernetes 26m Normal Created pod/payload-processing-e2e-ipp-b-bcc710-7478845cd-x7lwp Container created 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-dmk7s Container started 26m Normal Pulled pod/payload-processing-e2e-ipp-b-bcc710-7478845cd-x7lwp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-dmk7s Container created 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-dmk7s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-3b95dd-844587bbf8 Error creating: pods "payload-processing-e2e-ipp-b-3b95dd-844587bbf8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3b95dd: serviceaccount "payload-processing-e2e-ipp-b-3b95dd" not found 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f Error creating: pods "payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3b95dd: serviceaccount "payload-processing-e2e-ipp-b-3b95dd" not found 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-d05e21-openshift-default-6c99db878c Created pod: e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-d05e21-openshift-default-6c99db878c Created pod: e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-d05e21-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-d05e21-openshift-default-6c99db878c Created pod: e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-d05e21-openshift-default Scaled up replica set e2e-ipp-b-d05e21-openshift-default-6c99db878c from 1 to 4 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-3b95dd Scaled up replica set payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-3b95dd-844587bbf8 Created pod: payload-processing-e2e-ipp-b-3b95dd-844587bbf8-6rq9s 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-3b95dd Scaled up replica set payload-processing-e2e-ipp-b-3b95dd-844587bbf8 from 0 to 1 26m Normal Started pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv Container started 26m Normal Pulled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Pulled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j Container created 26m Normal Started pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp Container started 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f Created pod: payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-cwgl2 26m Normal Created pod/payload-processing-e2e-ipp-b-3b95dd-844587bbf8-6rq9s Container created 26m Normal Created pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp Container created 26m Normal Pulled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv Container created 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-cwgl2 Container started 26m Normal Started pod/payload-processing-e2e-ipp-b-3b95dd-844587bbf8-6rq9s Container started 26m Normal AddedInterface pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j Add eth0 [10.134.0.214/23] from ovn-kubernetes 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-cwgl2 Container created 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-cwgl2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j Container started 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-cwgl2 Add eth0 [10.132.0.244/23] from ovn-kubernetes 26m Normal AddedInterface pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv Add eth0 [10.133.1.43/23] from ovn-kubernetes 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-3b95dd-844587bbf8-6rq9s Add eth0 [10.133.1.44/23] from ovn-kubernetes 26m Normal AddedInterface pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt Add eth0 [10.132.0.243/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt Container created 26m Normal Started pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt Container started 26m Normal Pulled pod/payload-processing-e2e-ipp-b-3b95dd-844587bbf8-6rq9s Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Warning Unhealthy pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j Startup probe failed: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j Startup probe error: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused... 26m Warning ProbeError pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt Startup probe error: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv Startup probe failed: Get "http://10.133.1.43:15021/healthz/ready": dial tcp 10.133.1.43:15021: connect: connection refused 26m Warning Unhealthy pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt Startup probe failed: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv Startup probe error: Get "http://10.133.1.43:15021/healthz/ready": dial tcp 10.133.1.43:15021: connect: connection refused... 26m Warning ProbeError pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp Startup probe error: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp Startup probe failed: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-bcc710-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-bcc710-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-d05e21-5767b6546c-hmlcp Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-b-d05e21-6796c5fc4d-xzzzk Stopping container payload-processing 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-d05e21-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-d05e21-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Killing pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-jdpzn Stopping container istio-proxy 26m Normal Killing pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Stopping container istio-proxy 26m Warning ProbeError pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal Killing pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-cf754 Stopping container istio-proxy 26m Warning Unhealthy pod/e2e-shared-b-w1-e50e63-openshift-default-f89f68fdc-5zwp6 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Normal Killing pod/e2e-shared-a-w5-acefbb-openshift-default-7c99445d75-vr67b Stopping container istio-proxy 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-d05e21-openshift-default-6c99db878c Created pod: e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-d05e21-openshift-default-6c99db878c Created pod: e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f 26m Normal Killing pod/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf Stopping container payload-processing 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-d05e21-openshift-default New size: 6; reason: All metrics below target 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-d05e21-openshift-default Scaled up replica set e2e-ipp-b-d05e21-openshift-default-6c99db878c from 4 to 6 26m Normal Killing pod/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz Stopping container payload-pre-processing 26m Normal Started pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh Container started 26m Normal AddedInterface pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh Add eth0 [10.133.1.51/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f Container started 26m Normal Created pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh Container created 26m Normal AddedInterface pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f Add eth0 [10.134.0.228/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f Container created 26m Normal Pulled pod/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 Container created 26m Normal Started pod/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 Container started 26m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-ca229c-746c6c44f5-6j4pz Readiness probe failed: timeout: failed to connect service "10.134.0.52:50052" within 1s: context deadline exceeded 26m Warning Unhealthy pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh Startup probe failed: Get "http://10.133.1.51:15021/healthz/ready": dial tcp 10.133.1.51:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh Startup probe error: Get "http://10.133.1.51:15021/healthz/ready": dial tcp 10.133.1.51:15021: connect: connection refused... 26m Warning ProbeError pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Readiness probe error: Get "http://10.133.1.40:15021/healthz/ready": read tcp 10.133.0.2:51786->10.133.1.40:15021: read: connection reset by peer... 26m Warning Unhealthy pod/payload-processing-e2e-worker-w1-ca229c-5c5c6f9fc6-rvdqf Readiness probe failed: timeout: failed to connect service "10.134.0.53:50052" within 1s: context deadline exceeded 26m Warning Unhealthy pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f Startup probe failed: Get "http://10.134.0.228:15021/healthz/ready": dial tcp 10.134.0.228:15021: connect: connection refused 26m Warning Unhealthy pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Readiness probe failed: Get "http://10.133.1.40:15021/healthz/ready": read tcp 10.133.0.2:51786->10.133.1.40:15021: read: connection reset by peer 26m Warning ProbeError pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f Startup probe error: Get "http://10.134.0.228:15021/healthz/ready": dial tcp 10.134.0.228:15021: connect: connection refused... 26m Normal Created pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Container created 26m Normal Pulled pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Container started 25m Warning ProbeError pod/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 Startup probe error: Get "http://10.132.0.232:15021/healthz/ready": dial tcp 10.132.0.232:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 Startup probe failed: Get "http://10.132.0.232:15021/healthz/ready": dial tcp 10.132.0.232:15021: connect: connection refused 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f Created pod: e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-f0e8b9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Warning ProbeError pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Startup probe error: Get "http://10.133.1.40:15021/healthz/ready": dial tcp 10.133.1.40:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Startup probe failed: Get "http://10.133.1.40:15021/healthz/ready": dial tcp 10.133.1.40:15021: connect: connection refused 25m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 Stopping container payload-processing 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-f0e8b9-openshift-default Scaled up replica set e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f from 1 to 2 25m Normal Created pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx Container created 25m Normal AddedInterface pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx Add eth0 [10.132.1.3/23] from ovn-kubernetes 25m Normal Started pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx Container started 25m Normal Pulled pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-75443e-openshift-default Scaled down replica set e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc from 3 to 2 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-75443e-openshift-default New size: 2; reason: All metrics below target 25m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc Deleted pod: e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz 25m Normal Killing pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-7dxfz Stopping container istio-proxy 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-4be240-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-4be240-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-f1f206-openshift-default Scaled down replica set e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9 from 2 to 1 25m Normal Killing pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Stopping container istio-proxy 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-f1f206-openshift-default New size: 1; reason: All metrics below target 25m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9 Deleted pod: e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v 25m Warning Unhealthy pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx Startup probe failed: Get "http://10.132.1.3:15021/healthz/ready": dial tcp 10.132.1.3:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx Startup probe error: Get "http://10.132.1.3:15021/healthz/ready": dial tcp 10.132.1.3:15021: connect: connection refused... 25m Warning ProbeError pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-nl96v Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-90b406-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-90b406-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal SuccessfulDelete replicaset/e2e-shared-a-w4-e371c3-openshift-default-699697d895 Deleted pod: e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-e371c3-openshift-default New size: 1; reason: All metrics below target 25m Normal Killing pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-p9kqr Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e371c3-openshift-default Scaled down replica set e2e-shared-a-w4-e371c3-openshift-default-699697d895 from 2 to 1 25m Normal Killing pod/payload-pre-processing-e2e-ipp-b-4be240-84fd4f496c-7z9qv Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-ipp-b-4be240-58b8ddbdcb-n8p5d Stopping container payload-processing 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-4be240-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Killing pod/payload-processing-e2e-ipp-b-90b406-548d97d98d-tjsf4 Stopping container payload-processing 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-d05e21 Scaled up replica set payload-processing-e2e-ipp-b-d05e21-85db5c75d6 from 0 to 1 25m Normal Killing pod/payload-pre-processing-e2e-ipp-b-90b406-98b94bb7-4wb24 Stopping container payload-pre-processing 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-4be240-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767 Error creating: pods "payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d05e21: serviceaccount "payload-processing-e2e-ipp-b-d05e21" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-d05e21-85db5c75d6 Created pod: payload-processing-e2e-ipp-b-d05e21-85db5c75d6-kg8b9 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-d05e21-85db5c75d6 Error creating: pods "payload-processing-e2e-ipp-b-d05e21-85db5c75d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d05e21: serviceaccount "payload-processing-e2e-ipp-b-d05e21" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767 Created pod: payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-4j9qs 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-d05e21 Scaled up replica set payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767 from 0 to 1 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-4j9qs Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-1d6347-7778f9d655-545gn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-4j9qs Container created 25m Normal Started pod/payload-processing-e2e-ipp-b-d05e21-85db5c75d6-kg8b9 Container started 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-4j9qs Add eth0 [10.133.1.54/23] from ovn-kubernetes 25m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-1d6347-854b8c4d9d-zljd6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-d05e21-85db5c75d6-kg8b9 Add eth0 [10.132.1.4/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-4j9qs Container started 25m Normal Created pod/payload-processing-e2e-ipp-b-d05e21-85db5c75d6-kg8b9 Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-b-d05e21-85db5c75d6-kg8b9 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-90b406-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-90b406-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Killing pod/payload-processing-e2e-ipp-a-a1854b-58595bff5b-vvvvd Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a1854b-64bc599575-prc47 Stopping container payload-pre-processing 25m Normal Killing pod/e2e-shared-b-w5-1d6347-openshift-default-59d5cc8d8f-cjmm2 Stopping container istio-proxy 25m Normal Pulled pod/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t Container started 25m Normal Created pod/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t Container created 25m Normal Killing pod/e2e-ipp-a-a1854b-openshift-default-66ff8f6c6f-z8ff5 Stopping container istio-proxy 25m Warning Unhealthy pod/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t Startup probe failed: Get "http://10.133.1.42:15021/healthz/ready": dial tcp 10.133.1.42:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t Startup probe error: Get "http://10.133.1.42:15021/healthz/ready": dial tcp 10.133.1.42:15021: connect: connection refused... 24m Normal Killing pod/e2e-worker-w1-ca229c-openshift-default-549cb656c9-fbmff Stopping container istio-proxy 24m Normal Killing pod/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 Stopping container payload-pre-processing 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-396d70-openshift-default Scaled down replica set e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65 from 2 to 1 24m Normal SuccessfulDelete replicaset/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65 Deleted pod: e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc 24m Normal Killing pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-fwmdc Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-396d70-openshift-default New size: 1; reason: All metrics below target 24m Normal Killing pod/payload-pre-processing-e2e-ipp-b-bcc710-76656769cf-dmk7s Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-b-bcc710-7478845cd-x7lwp Stopping container payload-processing 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0efe43-openshift-default Scaled down replica set e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99 from 3 to 2 24m Normal Killing pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-0efe43-openshift-default New size: 2; reason: All metrics below target 24m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99 Deleted pod: e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-j5h4q 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-bcc710-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-bcc710-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-90b406 Scaled up replica set payload-processing-e2e-ipp-b-90b406-5f96f96db6 from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-90b406 Scaled up replica set payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667 from 0 to 1 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667 Error creating: pods "payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-90b406: serviceaccount "payload-processing-e2e-ipp-b-90b406" not found 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-90b406-5f96f96db6 Created pod: payload-processing-e2e-ipp-b-90b406-5f96f96db6-dwhwj 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-gfd5c Add eth0 [10.132.1.5/23] from ovn-kubernetes 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-gfd5c Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667 Created pod: payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-gfd5c 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-90b406-5f96f96db6 Error creating: pods "payload-processing-e2e-ipp-b-90b406-5f96f96db6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-90b406: serviceaccount "payload-processing-e2e-ipp-b-90b406" not found 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-gfd5c Container created 24m Normal Pulled pod/payload-processing-e2e-ipp-b-90b406-5f96f96db6-dwhwj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-gfd5c Container started 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-90b406-5f96f96db6-dwhwj Add eth0 [10.132.1.6/23] from ovn-kubernetes 24m Normal Started pod/payload-processing-e2e-ipp-b-90b406-5f96f96db6-dwhwj Container started 24m Normal Created pod/payload-processing-e2e-ipp-b-90b406-5f96f96db6-dwhwj Container created 24m Normal Killing pod/e2e-ipp-b-4be240-openshift-default-5b576754f5-9nph9 Stopping container istio-proxy 24m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-a7bb0c-5f9d6779f-jxjg9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-processing-e2e-worker-w5-a7bb0c-59c8dfbc9d-nvhdj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 Stopping container payload-pre-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-33c209-5b7b659959-97sdt Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-a-33c209-59b7c984c4-9wlm7 Stopping container payload-processing 24m Normal Killing pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Stopping container istio-proxy 24m Warning Unhealthy pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-shared-a-w2-396d70-openshift-default-59bc6b4c65-gxhnj Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/e2e-ipp-a-33c209-openshift-default-8dc5b9dc-gfl9m Stopping container istio-proxy 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-bcc710-75c6df84d Error creating: pods "payload-processing-e2e-ipp-b-bcc710-75c6df84d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bcc710: serviceaccount "payload-processing-e2e-ipp-b-bcc710" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-bcc710 Scaled up replica set payload-processing-e2e-ipp-b-bcc710-75c6df84d from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-bcc710 Scaled up replica set payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc from 0 to 1 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc Error creating: pods "payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bcc710: serviceaccount "payload-processing-e2e-ipp-b-bcc710" not found 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-bcc710-75c6df84d Created pod: payload-processing-e2e-ipp-b-bcc710-75c6df84d-h8h7n 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq Container created 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq Container started 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc Created pod: payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq Add eth0 [10.133.1.55/23] from ovn-kubernetes 24m Normal Started pod/payload-processing-e2e-ipp-b-bcc710-75c6df84d-h8h7n Container started 24m Normal Created pod/payload-processing-e2e-ipp-b-bcc710-75c6df84d-h8h7n Container created 24m Normal Pulled pod/payload-processing-e2e-ipp-b-bcc710-75c6df84d-h8h7n Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-bcc710-75c6df84d-h8h7n Add eth0 [10.133.1.56/23] from ovn-kubernetes 24m Normal Killing pod/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g Stopping container payload-pre-processing 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-396d70-56874f4975-w4bw5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-396d70-86964fb4d6-zmxfm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-6e91df-549c95f675-6zb6g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-6e91df-5cc45cccd4-5mt9c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/payload-processing-e2e-ipp-a-f0e8b9-78b48cfc98-7942f Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f0e8b9-6d979dbf9-jdqzn Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-6176c3-65cd57864b-tqgc2 Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-6176c3-564c7888cf-h2hqx Stopping container payload-pre-processing 23m Normal Killing pod/e2e-ipp-a-6176c3-openshift-default-64ddffcc7f-cl55j Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Stopping container istio-proxy 23m Warning ProbeError pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-worker-w5-a7bb0c-openshift-default-86bfdfb8bf-828n4 Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-g9fjc Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-b-w2-6e91df-openshift-default-8689d4c967-k6jj4 Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-d05e21-5d7d5cc767-4j9qs Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-d05e21-85db5c75d6-kg8b9 Stopping container payload-processing 23m Normal Killing pod/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j Stopping container payload-pre-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-2927fd-5cfc5f6ff9-p2hhp Stopping container payload-pre-processing 22m Normal Killing pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-2ms7f Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-brwmt Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-97f9j Stopping container istio-proxy 22m Normal Killing pod/payload-processing-e2e-ipp-a-2927fd-58947b47f4-wh6th Stopping container payload-processing 22m Normal Killing pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-t9mhp Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-h6lsv Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-b-d05e21-openshift-default-6c99db878c-xjqjh Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Stopping container istio-proxy 22m Normal Killing pod/payload-processing-e2e-shared-a-w0-9e3aa0-69c745ffb6-qrkwf Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-9e3aa0-6f999bc599-dsk2f Stopping container payload-pre-processing 22m Warning Unhealthy pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning ProbeError pod/e2e-ipp-a-2927fd-openshift-default-64dd8fc7d7-t6489 Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/payload-processing-e2e-worker-w2-da78c4-5bf784496d-rfcbt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-da78c4-8c5d878ff-8hb8j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-90b406-5dd68cb667-gfd5c Stopping container payload-pre-processing 22m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-b-90b406 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-90b406: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-90b406": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-90b406, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4cf7e7f9-3b15-46d2-a8b5-4c195fd5a67e, UID in object meta: 22m Normal Killing pod/payload-processing-e2e-ipp-b-90b406-5f96f96db6-dwhwj Stopping container payload-processing 22m Normal Killing pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-9nrsx Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-a-f0e8b9-openshift-default-6f75d78b5f-kq85z Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp Stopping container istio-proxy 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-a59ecb-openshift-default Scaled down replica set e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 from 2 to 1 22m Normal SuccessfulDelete replicaset/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9 Deleted pod: e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-kh8cp 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-a59ecb-openshift-default New size: 1; reason: All metrics below target 22m Normal Killing pod/e2e-ipp-b-90b406-openshift-default-68c56c8459-bgw27 Stopping container istio-proxy 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq Stopping container payload-pre-processing 22m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-b-bcc710-8c75455dc-6s2jq Readiness probe failed: dial tcp 10.133.1.55:9004: connect: connection refused 22m Normal Killing pod/payload-processing-e2e-ipp-b-bcc710-75c6df84d-h8h7n Stopping container payload-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-3b95dd-844587bbf8-6rq9s Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-3b95dd-6969b8789f-cwgl2 Stopping container payload-pre-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v Stopping container payload-processing 22m Normal Killing pod/e2e-ipp-b-bcc710-openshift-default-645587584f-q2m2t Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-b-3b95dd-openshift-default-75fccfd4c9-c7jlr Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w0-9e3aa0-openshift-default-bc5d75d8f-jp6fs Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Stopping container istio-proxy 21m Normal Killing pod/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc Stopping container payload-pre-processing 21m Warning Unhealthy pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-worker-w2-da78c4-openshift-default-84dbb4c47f-pfpzv Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc Stopping container payload-processing 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-e371c3-67c8f8ffbc-8qcxh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-e371c3-5b6c6695c6-hkq5v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-shared-a-w4-e371c3-openshift-default-699697d895-thj9g Stopping container istio-proxy 21m Normal Killing pod/payload-processing-e2e-shared-b-w0-f1f206-5bc75d57d8-kb9kl Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-f1f206-dbc6955ff-k76fk Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs Stopping container payload-pre-processing 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-5704d6-695bc45c86-nhwbc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-5704d6-68bd9c5899-75mlp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-shared-b-w0-f1f206-openshift-default-d9b5df6f9-lgvxm Stopping container istio-proxy 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-75443e-6b6cd45ccb-fnql4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-75443e-7c475f6fbc-d7wdc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-24sv2 Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Stopping container istio-proxy 21m Warning ProbeError pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-shared-a-w6-75443e-openshift-default-567ff8b6bc-22r8m Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Killing pod/e2e-shared-a-w3-5704d6-openshift-default-84b89c8cb9-gwzxq Stopping container istio-proxy 21m Normal Killing pod/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 Stopping container payload-processing 21m Normal Killing pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Stopping container istio-proxy 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-a59ecb-dfbdf6cdc-zsbgs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw Stopping container payload-processing 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-a59ecb-99d68b8b4-zwhsn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 Stopping container payload-pre-processing 21m Warning ProbeError pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-shared-b-w4-a59ecb-openshift-default-6ffc4474c9-bzj5h Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning Unhealthy pod/payload-processing-e2e-worker-w0-2df0ab-5649f578f6-lbf49 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn Stopping container payload-pre-processing 21m Normal Killing pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-nz2js Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-b-w6-0efe43-openshift-default-6b8ffb8d99-4lg9b Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-b-w3-228454-openshift-default-6476dbf645-pwj4b Stopping container istio-proxy 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-0efe43-7474755dbc-nqdc2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-0efe43-54c6bd4ff6-pdj66 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-2df0ab-6bb6d6f87f-4sqjj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-228454-df8d85c7d-jzcbw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-228454-bcfd9db47-bzmp7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-processing-e2e-worker-w3-00926c-88774844d-mz5bp Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-worker-w3-00926c-77f9968595-6zj5z Stopping container payload-pre-processing 20m Normal Killing pod/e2e-worker-w0-2df0ab-openshift-default-757d5db88c-s268t Stopping container istio-proxy 20m Normal Killing pod/payload-pre-processing-e2e-worker-w6-39529e-7fcb96875-rx55z Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-worker-w6-39529e-79cf96b4cb-qthk4 Stopping container payload-processing 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-46b13b-56878dd6bc-stcrn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-worker-w4-46b13b-646fb546c8-4xsd2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Stopping container istio-proxy 20m Warning ProbeError pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-worker-w4-46b13b-openshift-default-7b98ffbf7c-b4j8n Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Stopping container istio-proxy 20m Normal Killing pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Stopping container istio-proxy 20m Warning ProbeError pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-worker-w3-00926c-openshift-default-5b8b9f5575-7pnxf Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning Unhealthy pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-worker-w6-39529e-openshift-default-cb57f9db7-m75qd Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-68bcg Stopping container discovery 20m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 20m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 20m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-68bcg 20m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Add eth0 [10.134.1.1/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-52g42 20m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-cl2qx 20m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 Add eth0 [10.133.1.64/23] from ovn-kubernetes 20m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Add eth0 [10.133.1.65/23] from ovn-kubernetes 20m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 20m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 1 to 4 20m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 20m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 Container started 20m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Container created 20m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Container started 20m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 Container created 20m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Container started 20m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Container created 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 Startup probe failed: Get "http://10.133.1.64:15021/healthz/ready": dial tcp 10.133.1.64:15021: connect: connection refused 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Startup probe failed: Get "http://10.134.1.1:15021/healthz/ready": dial tcp 10.134.1.1:15021: connect: connection refused 19m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Startup probe error: Get "http://10.134.1.1:15021/healthz/ready": dial tcp 10.134.1.1:15021: connect: connection refused... 19m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-d9pw9 Startup probe error: Get "http://10.133.1.64:15021/healthz/ready": dial tcp 10.133.1.64:15021: connect: connection refused... 19m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Startup probe error: Get "http://10.133.1.65:15021/healthz/ready": dial tcp 10.133.1.65:15021: connect: connection refused... 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Startup probe failed: Get "http://10.133.1.65:15021/healthz/ready": dial tcp 10.133.1.65:15021: connect: connection refused 19m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: 19m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 4 to 5 19m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-gqq8b 19m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Add eth0 [10.134.1.3/23] from ovn-kubernetes 19m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Container started 19m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Container created 19m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Startup probe failed: Get "http://10.134.1.3:15021/healthz/ready": dial tcp 10.134.1.3:15021: connect: connection refused 19m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Startup probe error: Get "http://10.134.1.3:15021/healthz/ready": dial tcp 10.134.1.3:15021: connect: connection refused... 18m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 5 to 10 18m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 18m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-d7k5x 18m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 18m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Container started 18m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Add eth0 [10.132.1.13/23] from ovn-kubernetes 18m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Container created 18m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Container started 18m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Add eth0 [10.134.1.5/23] from ovn-kubernetes 18m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Container started 18m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Container created 18m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Add eth0 [10.133.1.68/23] from ovn-kubernetes 18m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Container started 18m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Container created 18m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Add eth0 [10.132.1.14/23] from ovn-kubernetes 18m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Container started 18m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Container created 18m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Add eth0 [10.133.1.67/23] from ovn-kubernetes 18m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Container created 18m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Startup probe error: Get "http://10.133.1.68:15021/healthz/ready": dial tcp 10.133.1.68:15021: connect: connection refused... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Startup probe error: Get "http://10.133.1.67:15021/healthz/ready": dial tcp 10.133.1.67:15021: connect: connection refused... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Startup probe error: Get "http://10.132.1.14:15021/healthz/ready": dial tcp 10.132.1.14:15021: connect: connection refused... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Startup probe failed: Get "http://10.132.1.14:15021/healthz/ready": dial tcp 10.132.1.14:15021: connect: connection refused 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Startup probe failed: Get "http://10.133.1.67:15021/healthz/ready": dial tcp 10.133.1.67:15021: connect: connection refused 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Startup probe error: Get "http://10.134.1.5:15021/healthz/ready": dial tcp 10.134.1.5:15021: connect: connection refused... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Startup probe failed: Get "http://10.134.1.5:15021/healthz/ready": dial tcp 10.134.1.5:15021: connect: connection refused 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Startup probe failed: Get "http://10.132.1.13:15021/healthz/ready": dial tcp 10.132.1.13:15021: connect: connection refused 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Startup probe error: Get "http://10.132.1.13:15021/healthz/ready": dial tcp 10.132.1.13:15021: connect: connection refused... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Startup probe failed: Get "http://10.133.1.68:15021/healthz/ready": dial tcp 10.133.1.68:15021: connect: connection refused 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Readiness probe failed: Get "http://10.133.1.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-52g42 Readiness probe error: Get "http://10.133.1.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Readiness probe failed: Get "http://10.134.1.3:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-gqq8b Readiness probe error: Get "http://10.134.1.3:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Readiness probe error: Get "http://10.134.1.1:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-cl2qx Readiness probe failed: Get "http://10.134.1.1:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Startup probe failed: Get "http://10.134.1.5:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Startup probe error: Get "http://10.134.1.5:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Startup probe error: Get "http://10.133.1.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Startup probe error: Get "http://10.132.1.13:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Startup probe error: Get "http://10.133.1.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-f2kp6 Startup probe failed: Get "http://10.132.1.13:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-p5gtn Startup probe failed: Get "http://10.133.1.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-fb779 Startup probe failed: Get "http://10.133.1.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Startup probe failed: Get "http://10.132.1.14:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-kxfp8 Startup probe error: Get "http://10.132.1.14:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Readiness probe error: Get "http://10.134.1.5:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-d7k5x Readiness probe failed: Get "http://10.134.1.5:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 14m Normal AddedLabel gateway/e2e-worker-single-main-d3d611 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-d3d611 14m Normal SuccessfulCreate replicaset/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f Created pod: e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx 14m Normal ScalingReplicaSet deployment/e2e-worker-single-main-d3d611-openshift-default Scaled up replica set e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f from 0 to 1 13m Normal AddedInterface pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Add eth0 [10.134.1.17/23] from ovn-kubernetes 13m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-d3d611-667f65b9b6 Error creating: pods "payload-processing-e2e-worker-single-main-d3d611-667f65b9b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-d3d611: serviceaccount "payload-processing-e2e-worker-single-main-d3d611" not found 13m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-d3d611-8589b86f7f Created pod: payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 13m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-d3d611-667f65b9b6 Created pod: payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n 13m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-d3d611 Scaled up replica set payload-pre-processing-e2e-worker-single-main-d3d611-8589b86f7f from 0 to 1 13m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-d3d611-8589b86f7f Error creating: pods "payload-pre-processing-e2e-worker-single-main-d3d611-8589b86f7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-d3d611: serviceaccount "payload-processing-e2e-worker-single-main-d3d611" not found 13m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-d3d611 Scaled up replica set payload-processing-e2e-worker-single-main-d3d611-667f65b9b6 from 0 to 1 13m Normal Created pod/payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 Container created 13m Normal Created pod/payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n Container created 13m Normal Pulled pod/payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 13m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 13m Normal Started pod/payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 Container started 13m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n Add eth0 [10.134.1.19/23] from ovn-kubernetes 13m Normal Started pod/payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n Container started 13m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 Add eth0 [10.134.1.18/23] from ovn-kubernetes 13m Warning Unhealthy pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Readiness probe failed: Get "http://10.134.1.17:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning ProbeError pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Readiness probe error: Get "http://10.134.1.17:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Normal Started pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Container started 13m Normal Created pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Container created 13m Normal Pulled pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Warning Unhealthy pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Startup probe failed: Get "http://10.134.1.17:15021/healthz/ready": dial tcp 10.134.1.17:15021: connect: connection refused 13m Warning ProbeError pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Startup probe error: Get "http://10.134.1.17:15021/healthz/ready": dial tcp 10.134.1.17:15021: connect: connection refused... 13m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-d3d611-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 13m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-d3d611-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 12m Normal Killing pod/payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n Stopping container payload-processing 12m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 Stopping container payload-pre-processing 12m Normal Killing pod/e2e-worker-single-main-d3d611-openshift-default-55fb7bcb8f6z2tx Stopping container istio-proxy 12m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-d3d611-667f65b9bwtw5n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 12m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-d3d611-8589btn886 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Normal ScalingReplicaSet deployment/e2e-worker-single-main-2ef0c7-openshift-default Scaled up replica set e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5 from 0 to 1 10m Normal AddedLabel gateway/e2e-worker-single-main-2ef0c7 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-2ef0c7 10m Normal SuccessfulCreate replicaset/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5 Created pod: e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 10m Normal AddedInterface pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Add eth0 [10.134.1.26/23] from ovn-kubernetes 9m46s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-2ef0c7 Scaled up replica set payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cd from 0 to 1 9m46s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cd Error creating: pods "payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2ef0c7: serviceaccount "payload-processing-e2e-worker-single-main-2ef0c7" not found 9m46s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-2ef0c7 Scaled up replica set payload-pre-processing-e2e-worker-single-main-2ef0c7-7466dc6879 from 0 to 1 9m46s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466dc6879 Error creating: pods "payload-pre-processing-e2e-worker-single-main-2ef0c7-7466dc6879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2ef0c7: serviceaccount "payload-processing-e2e-worker-single-main-2ef0c7" not found 9m46s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466dc6879 Created pod: payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j 9m45s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 9m45s Normal Created pod/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm Container created 9m45s Normal Created pod/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j Container created 9m45s Normal Started pod/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm Container started 9m45s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j Add eth0 [10.134.1.28/23] from ovn-kubernetes 9m45s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cd Created pod: payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm 9m45s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm Add eth0 [10.134.1.29/23] from ovn-kubernetes 9m45s Normal Pulled pod/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 9m45s Normal Started pod/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j Container started 9m32s Normal Pulled pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 9m32s Normal Created pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Container created 9m32s Normal Started pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Container started 9m27s Warning ProbeError pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Startup probe error: Get "http://10.134.1.26:15021/healthz/ready": dial tcp 10.134.1.26:15021: connect: connection refused... 9m27s Warning Unhealthy pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Startup probe failed: Get "http://10.134.1.26:15021/healthz/ready": dial tcp 10.134.1.26:15021: connect: connection refused 9m16s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-2ef0c7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m16s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-2ef0c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m1s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-2ef0c7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 9m1s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-2ef0c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m39s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j Stopping container payload-pre-processing 8m39s Normal Killing pod/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm Stopping container payload-processing 8m28s Normal Killing pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Stopping container istio-proxy 8m25s Warning Unhealthy pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Readiness probe failed: HTTP probe failed with statuscode: 503 8m25s Warning ProbeError pod/e2e-worker-single-main-2ef0c7-openshift-default-6f74b859b5lxdg2 Readiness probe error: HTTP probe failed with statuscode: 503... 8m23s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-2ef0c7-6f45f67cdvqcsm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m22s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-2ef0c7-7466df4x8j Readiness probe failed: service unhealthy (responded with "NOT_SERVING")