=== events in openshift-ingress (2026-10-30T20:03:09Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 35m Normal Scheduled pod/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-8f9j5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-8f9j5 to ip-10-0-130-164.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-api-b-8e6159-647c947c95-tdxmr Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-8e6159-647c947c95-tdxmr to ip-10-0-132-19.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Successfully assigned openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b to ip-10-0-132-19.us-east-2.compute.internal 16m Normal Scheduled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d Successfully assigned openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l Successfully assigned openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l to ip-10-0-137-200.us-east-2.compute.internal 18m Warning FailedScheduling pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Successfully assigned openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r to ip-10-0-132-19.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z Successfully assigned openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z to ip-10-0-130-164.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj Successfully assigned openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj to ip-10-0-132-19.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj Successfully assigned openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj to ip-10-0-130-164.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh Successfully assigned openshift-ingress/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq Successfully assigned openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq to ip-10-0-130-164.us-east-2.compute.internal 18m Normal Scheduled pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Successfully assigned openshift-ingress/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr to ip-10-0-132-19.us-east-2.compute.internal 18m Warning FailedScheduling pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Successfully assigned openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 to ip-10-0-130-164.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Successfully assigned openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 to ip-10-0-132-19.us-east-2.compute.internal 17m Normal Scheduled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp Successfully assigned openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Successfully assigned openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 to ip-10-0-137-200.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk to ip-10-0-130-164.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n to ip-10-0-137-200.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr Successfully assigned openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr to ip-10-0-132-19.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-ait-7c74535f-openshift-default-f55494c4f-fwpmp Successfully assigned openshift-ingress/e2e-ait-7c74535f-openshift-default-f55494c4f-fwpmp to ip-10-0-132-19.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr to ip-10-0-132-19.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw to ip-10-0-130-164.us-east-2.compute.internal 57m Normal Scheduled pod/router-default-ddbc459d5-jmrh9 Successfully assigned openshift-ingress/router-default-ddbc459d5-jmrh9 to ip-10-0-130-164.us-east-2.compute.internal 57m Warning FailedScheduling pod/router-default-ddbc459d5-jmrh9 no nodes available to schedule pods 62m Warning FailedScheduling pod/router-default-ddbc459d5-jmrh9 no nodes available to schedule pods 40m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk to ip-10-0-137-200.us-east-2.compute.internal 23m Normal Scheduled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Successfully assigned openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv to ip-10-0-137-200.us-east-2.compute.internal 22m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 Successfully assigned openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 to ip-10-0-130-164.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Successfully assigned openshift-ingress/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr to ip-10-0-130-164.us-east-2.compute.internal 22m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Successfully assigned openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d to ip-10-0-132-19.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 to ip-10-0-132-19.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Successfully assigned openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp to ip-10-0-130-164.us-east-2.compute.internal 23m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Successfully assigned openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v to ip-10-0-132-19.us-east-2.compute.internal 23m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Normal Scheduled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Successfully assigned openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj to ip-10-0-137-200.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 Successfully assigned openshift-ingress/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 to ip-10-0-130-164.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs Successfully assigned openshift-ingress/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs to ip-10-0-130-164.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq to ip-10-0-137-200.us-east-2.compute.internal 23m Warning FailedScheduling pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Normal Scheduled pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 Successfully assigned openshift-ingress/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 to ip-10-0-130-164.us-east-2.compute.internal 19m Normal Scheduled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm Successfully assigned openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm to ip-10-0-130-164.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Successfully assigned openshift-ingress/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k to ip-10-0-132-19.us-east-2.compute.internal 19m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 36m Normal Scheduled pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Successfully assigned openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs to ip-10-0-130-164.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds Successfully assigned openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds to ip-10-0-132-19.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 Successfully assigned openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 to ip-10-0-130-164.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h to ip-10-0-137-200.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h Successfully assigned openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h to ip-10-0-137-200.us-east-2.compute.internal 22m Warning FailedScheduling pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Successfully assigned openshift-ingress/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd to ip-10-0-130-164.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 Successfully assigned openshift-ingress/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 to ip-10-0-130-164.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Warning FailedScheduling pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 Successfully assigned openshift-ingress/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 to ip-10-0-132-19.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 to ip-10-0-137-200.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Normal Scheduled pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Successfully assigned openshift-ingress/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw to ip-10-0-132-19.us-east-2.compute.internal 18m Warning FailedScheduling pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Successfully assigned openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 to ip-10-0-130-164.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Successfully assigned openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk to ip-10-0-130-164.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq Successfully assigned openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq to ip-10-0-137-200.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 to ip-10-0-132-19.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 Successfully assigned openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 to ip-10-0-130-164.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Successfully assigned openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r to ip-10-0-132-19.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Successfully assigned openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j to ip-10-0-132-19.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 Successfully assigned openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 to ip-10-0-130-164.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Normal Scheduled pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 Successfully assigned openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 to ip-10-0-137-200.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 to ip-10-0-137-200.us-east-2.compute.internal 33m Warning FailedScheduling pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Warning FailedScheduling pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Normal Scheduled pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Successfully assigned openshift-ingress/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c to ip-10-0-130-164.us-east-2.compute.internal 18m Warning FailedScheduling pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 Successfully assigned openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 to ip-10-0-130-164.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Successfully assigned openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc to ip-10-0-132-19.us-east-2.compute.internal 19m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-67f4m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-67f4m 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-7ckdm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 to ip-10-0-137-200.us-east-2.compute.internal 8m36s Normal Scheduled pod/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz to ip-10-0-130-164.us-east-2.compute.internal 5m18s Normal Scheduled pod/payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp to ip-10-0-130-164.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 to ip-10-0-130-164.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 to ip-10-0-132-19.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn to ip-10-0-132-19.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp to ip-10-0-132-19.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 to ip-10-0-132-19.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s to ip-10-0-137-200.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s to ip-10-0-132-19.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Successfully assigned openshift-ingress/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf to ip-10-0-132-19.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr to ip-10-0-130-164.us-east-2.compute.internal 23m Warning FailedScheduling pod/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 27m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 to ip-10-0-130-164.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp to ip-10-0-132-19.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk to ip-10-0-130-164.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d to ip-10-0-137-200.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 to ip-10-0-137-200.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-mt-48b1a6a7-5547c65796-crcbs Successfully assigned openshift-ingress/payload-processing-e2e-mt-48b1a6a7-5547c65796-crcbs to ip-10-0-130-164.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69 Successfully assigned openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69 to ip-10-0-137-200.us-east-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt Successfully assigned openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt to ip-10-0-137-200.us-east-2.compute.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-j2hcl Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-j2hcl to ip-10-0-130-164.us-east-2.compute.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-nxjsk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-nxjsk to ip-10-0-132-19.us-east-2.compute.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-be6b74-7c46964458-qddjb Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-be6b74-7c46964458-qddjb to ip-10-0-130-164.us-east-2.compute.internal 15m Normal Scheduled pod/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-pmhnj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-pmhnj to ip-10-0-130-164.us-east-2.compute.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-aa49c5-c48688486-ljmsd Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-aa49c5-c48688486-ljmsd to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Successfully assigned openshift-ingress/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt to ip-10-0-132-19.us-east-2.compute.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-htnjx Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-htnjx to ip-10-0-132-19.us-east-2.compute.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-1a844e-59d5b45d66-rjhs5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1a844e-59d5b45d66-rjhs5 to ip-10-0-132-19.us-east-2.compute.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn to ip-10-0-137-200.us-east-2.compute.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 to ip-10-0-132-19.us-east-2.compute.internal 21m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-e50648-85599ff484-jnxb9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e50648-85599ff484-jnxb9 to ip-10-0-132-19.us-east-2.compute.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 to ip-10-0-130-164.us-east-2.compute.internal 19m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 to ip-10-0-132-19.us-east-2.compute.internal 18m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 17m Normal Scheduled pod/payload-processing-e2e-ipp-a-c2b092-54548c6cf4-2v2zd Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c2b092-54548c6cf4-2v2zd to ip-10-0-130-164.us-east-2.compute.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-a-826518-7c7d6889c4-8gkdn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-826518-7c7d6889c4-8gkdn to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb to ip-10-0-132-19.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-derive-c5ed0670-openshift-default-5f977cdc49-rqrgb Successfully assigned openshift-ingress/e2e-derive-c5ed0670-openshift-default-5f977cdc49-rqrgb to ip-10-0-132-19.us-east-2.compute.internal 17m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 17m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n to ip-10-0-137-200.us-east-2.compute.internal 23m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm to ip-10-0-130-164.us-east-2.compute.internal 21m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj to ip-10-0-137-200.us-east-2.compute.internal 19m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 29m Warning FailedScheduling pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Warning FailedScheduling pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Normal Scheduled pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c Successfully assigned openshift-ingress/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-a-7aa217-7bc848b57b-psqwk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7bc848b57b-psqwk to ip-10-0-130-164.us-east-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 to ip-10-0-132-19.us-east-2.compute.internal 33m Warning FailedScheduling pod/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Warning FailedScheduling pod/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 31m Normal Scheduled pod/payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 to ip-10-0-132-19.us-east-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 to ip-10-0-137-200.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-inf-b-76117c-7b6dc4bd8-tjgr5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-76117c-7b6dc4bd8-tjgr5 to ip-10-0-132-19.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz Successfully assigned openshift-ingress/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz to ip-10-0-130-164.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-b-69f3a9-765dc684fd-8rp54 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-69f3a9-765dc684fd-8rp54 to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-api-a-f4012a-b58495cbc-zdz5w Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-f4012a-b58495cbc-zdz5w to ip-10-0-132-19.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb to ip-10-0-130-164.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq to ip-10-0-130-164.us-east-2.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw to ip-10-0-132-19.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Successfully assigned openshift-ingress/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 to ip-10-0-132-19.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh to ip-10-0-130-164.us-east-2.compute.internal 33m Warning FailedScheduling pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Warning FailedScheduling pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5 Successfully assigned openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5 to ip-10-0-137-200.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-api-b-f61e62-5d44b7544d-zkr4f Successfully assigned openshift-ingress/payload-processing-e2e-api-b-f61e62-5d44b7544d-zkr4f to ip-10-0-130-164.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-api-b-9b2e27-56c4c6f766-fphsc Successfully assigned openshift-ingress/payload-processing-e2e-api-b-9b2e27-56c4c6f766-fphsc to ip-10-0-130-164.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-api-b-8e6159-5fd5bcf88d-mmxjf Successfully assigned openshift-ingress/payload-processing-e2e-api-b-8e6159-5fd5bcf88d-mmxjf to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-api-a-f4012a-78c8dd4c94-sgr5m Successfully assigned openshift-ingress/payload-processing-e2e-api-a-f4012a-78c8dd4c94-sgr5m to ip-10-0-132-19.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-api-a-9a718a-6f7445cb4d-h2hxs Successfully assigned openshift-ingress/payload-processing-e2e-api-a-9a718a-6f7445cb4d-h2hxs to ip-10-0-130-164.us-east-2.compute.internal 33m Warning FailedScheduling pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Warning FailedScheduling pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Normal Scheduled pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Successfully assigned openshift-ingress/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-api-a-8ceda0-6ff4cc8588-xbn7c Successfully assigned openshift-ingress/payload-processing-e2e-api-a-8ceda0-6ff4cc8588-xbn7c to ip-10-0-132-19.us-east-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc Successfully assigned openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc to ip-10-0-137-200.us-east-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp Successfully assigned openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp to ip-10-0-137-200.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf to ip-10-0-137-200.us-east-2.compute.internal 21m Warning FailedScheduling pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Warning FailedScheduling pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Normal Scheduled pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 Successfully assigned openshift-ingress/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 to ip-10-0-130-164.us-east-2.compute.internal 23m Warning FailedScheduling pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Warning FailedScheduling pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Successfully assigned openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 to ip-10-0-137-200.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-5f78687d5b-gblqx Successfully assigned openshift-ingress/payload-processing-5f78687d5b-gblqx to ip-10-0-132-19.us-east-2.compute.internal 21m Normal Scheduled pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg Successfully assigned openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg to ip-10-0-137-200.us-east-2.compute.internal 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-7ckdm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-9nptk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-9nptk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-c2jdw 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-c2jdw 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-dtvch 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-dtvch 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-g89kd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c to ip-10-0-137-200.us-east-2.compute.internal 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-g89kd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 32m Normal Scheduled pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd Successfully assigned openshift-ingress/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd to ip-10-0-132-19.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Successfully assigned openshift-ingress/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z to ip-10-0-132-19.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Normal Scheduled pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Successfully assigned openshift-ingress/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx to ip-10-0-132-19.us-east-2.compute.internal 19m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-rpbzh 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-rpbzh 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh to ip-10-0-137-200.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Successfully assigned openshift-ingress/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt to ip-10-0-130-164.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Normal Scheduled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw Successfully assigned openshift-ingress/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw to ip-10-0-130-164.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-8fk7t 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-8fk7t 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Normal Scheduled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c Successfully assigned openshift-ingress/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c to ip-10-0-130-164.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 Successfully assigned openshift-ingress/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 to ip-10-0-130-164.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb to ip-10-0-137-200.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Successfully assigned openshift-ingress/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt to ip-10-0-132-19.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Successfully assigned openshift-ingress/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v to ip-10-0-132-19.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd Successfully assigned openshift-ingress/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd to ip-10-0-130-164.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-pgdtb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-pgdtb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-s5pxv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-s5pxv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm Successfully assigned openshift-ingress/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm to ip-10-0-130-164.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd to ip-10-0-137-200.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 31m Normal Scheduled pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Successfully assigned openshift-ingress/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 to ip-10-0-132-19.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Successfully assigned openshift-ingress/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn to ip-10-0-132-19.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Successfully assigned openshift-ingress/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 to ip-10-0-130-164.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x Successfully assigned openshift-ingress/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x to ip-10-0-130-164.us-east-2.compute.internal 29m Warning FailedScheduling pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Successfully assigned openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km to ip-10-0-137-200.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Successfully assigned openshift-ingress/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 to ip-10-0-130-164.us-east-2.compute.internal 29m Warning FailedScheduling pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 29m Normal Scheduled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 Successfully assigned openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 to ip-10-0-130-164.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 Successfully assigned openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 to ip-10-0-130-164.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Successfully assigned openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt to ip-10-0-132-19.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq Successfully assigned openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq to ip-10-0-137-200.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d to ip-10-0-137-200.us-east-2.compute.internal 19m Normal Scheduled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg Successfully assigned openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Successfully assigned openshift-ingress/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 to ip-10-0-130-164.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n Successfully assigned openshift-ingress/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n to ip-10-0-130-164.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p to ip-10-0-132-19.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 to ip-10-0-132-19.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp to ip-10-0-130-164.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 to ip-10-0-132-19.us-east-2.compute.internal 8m36s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c to ip-10-0-132-19.us-east-2.compute.internal 5m19s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp to ip-10-0-132-19.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt to ip-10-0-130-164.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-z9mj5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-z9mj5 to ip-10-0-132-19.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 to ip-10-0-137-200.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx to ip-10-0-132-19.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 to ip-10-0-130-164.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz to ip-10-0-132-19.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt to ip-10-0-132-19.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-pc9sr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-pc9sr to ip-10-0-132-19.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 to ip-10-0-137-200.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr to ip-10-0-132-19.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h to ip-10-0-130-164.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g to ip-10-0-137-200.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk to ip-10-0-137-200.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-rmdbz Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-rmdbz to ip-10-0-130-164.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl to ip-10-0-137-200.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Successfully assigned openshift-ingress/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x to ip-10-0-130-164.us-east-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk to ip-10-0-137-200.us-east-2.compute.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-vtdcl Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-vtdcl to ip-10-0-130-164.us-east-2.compute.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-tflx2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-tflx2 to ip-10-0-132-19.us-east-2.compute.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-wpmft Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-wpmft to ip-10-0-130-164.us-east-2.compute.internal 15m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-jmntj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-jmntj to ip-10-0-132-19.us-east-2.compute.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-nt6th Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-nt6th to ip-10-0-132-19.us-east-2.compute.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-vcb79 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-vcb79 to ip-10-0-132-19.us-east-2.compute.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-k5s8s Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-k5s8s to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-t442z Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-t442z to ip-10-0-132-19.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Successfully assigned openshift-ingress/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 to ip-10-0-132-19.us-east-2.compute.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-k5stc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-k5stc to ip-10-0-132-19.us-east-2.compute.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 to ip-10-0-137-200.us-east-2.compute.internal 19m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 to ip-10-0-130-164.us-east-2.compute.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 to ip-10-0-132-19.us-east-2.compute.internal 18m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-zf2r5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-zf2r5 to ip-10-0-132-19.us-east-2.compute.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh to ip-10-0-137-200.us-east-2.compute.internal 17m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-826518-5879559bc5-tf7wf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-5879559bc5-tf7wf to ip-10-0-132-19.us-east-2.compute.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss to ip-10-0-132-19.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-chtgk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-chtgk to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-rqhfz Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-rqhfz to ip-10-0-130-164.us-east-2.compute.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-jphs8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-jphs8 to ip-10-0-130-164.us-east-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q to ip-10-0-137-200.us-east-2.compute.internal 33m Warning FailedScheduling pod/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 31m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 to ip-10-0-132-19.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Successfully assigned openshift-ingress/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 to ip-10-0-130-164.us-east-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg to ip-10-0-132-19.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd to ip-10-0-132-19.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn to ip-10-0-132-19.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp to ip-10-0-132-19.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz to ip-10-0-130-164.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh to ip-10-0-132-19.us-east-2.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr Successfully assigned openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr to ip-10-0-137-200.us-east-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j to ip-10-0-132-19.us-east-2.compute.internal 33m Warning FailedScheduling pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Warning FailedScheduling pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j to ip-10-0-137-200.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-mjqtx Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-mjqtx to ip-10-0-130-164.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf Successfully assigned openshift-ingress/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf to ip-10-0-130-164.us-east-2.compute.internal 51m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-68d996c5577pfxb Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-68d996c5577pfxb to ip-10-0-130-164.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f to ip-10-0-132-19.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-api-a-9a718a-67df85b967-xbx45 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-9a718a-67df85b967-xbx45 to ip-10-0-132-19.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-api-a-8ceda0-54c896b567-45659 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-8ceda0-54c896b567-45659 to ip-10-0-130-164.us-east-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-mbjb4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-mbjb4 to ip-10-0-132-19.us-east-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl to ip-10-0-137-200.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l to ip-10-0-130-164.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc to ip-10-0-137-200.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Successfully assigned openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz to ip-10-0-132-19.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Successfully assigned openshift-ingress/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d to ip-10-0-132-19.us-east-2.compute.internal 22m Warning FailedScheduling pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Warning FailedScheduling pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Normal Scheduled pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 Successfully assigned openshift-ingress/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 to ip-10-0-132-19.us-east-2.compute.internal 23m Warning FailedScheduling pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Normal Scheduled pod/payload-pre-processing-7b98bb678c-svc7x Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-svc7x to ip-10-0-132-19.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 to ip-10-0-137-200.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl to ip-10-0-132-19.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw to ip-10-0-132-19.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r to ip-10-0-130-164.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Successfully assigned openshift-ingress/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 to ip-10-0-130-164.us-east-2.compute.internal 19m Normal Scheduled pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql Successfully assigned openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql to ip-10-0-137-200.us-east-2.compute.internal 19m Warning FailedScheduling pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl to ip-10-0-132-19.us-east-2.compute.internal 17m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 to ip-10-0-137-200.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb to ip-10-0-137-200.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 16m Normal Scheduled pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Successfully assigned openshift-ingress/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf to ip-10-0-132-19.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp to ip-10-0-137-200.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb to ip-10-0-130-164.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g to ip-10-0-137-200.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s to ip-10-0-132-19.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr to ip-10-0-130-164.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs to ip-10-0-130-164.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj to ip-10-0-130-164.us-east-2.compute.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-xlxqw to ip-10-0-132-19.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 to ip-10-0-132-19.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Successfully assigned openshift-ingress/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz to ip-10-0-130-164.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d to ip-10-0-137-200.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 to ip-10-0-132-19.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-wz7wl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-wz7wl to ip-10-0-132-19.us-east-2.compute.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-vs4nx to ip-10-0-130-164.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg to ip-10-0-130-164.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Normal Scheduled pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Successfully assigned openshift-ingress/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq to ip-10-0-132-19.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw Successfully assigned openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw to ip-10-0-132-19.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-v9tpn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-v9tpn to ip-10-0-130-164.us-east-2.compute.internal 16m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 to ip-10-0-137-200.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn to ip-10-0-130-164.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs to ip-10-0-130-164.us-east-2.compute.internal 16m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr to ip-10-0-132-19.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-x4nvl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Warning FailedScheduling pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-x4nvl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Successfully assigned openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv to ip-10-0-130-164.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb to ip-10-0-132-19.us-east-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt to ip-10-0-137-200.us-east-2.compute.internal 16m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr to ip-10-0-130-164.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj to ip-10-0-132-19.us-east-2.compute.internal 18m Warning FailedScheduling pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Normal Scheduled pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Successfully assigned openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 to ip-10-0-130-164.us-east-2.compute.internal 33m Warning FailedScheduling pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-pthrg to ip-10-0-132-19.us-east-2.compute.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-m5mbs to ip-10-0-132-19.us-east-2.compute.internal 19m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 to ip-10-0-132-19.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d Successfully assigned openshift-ingress/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d to ip-10-0-130-164.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Warning FailedScheduling pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x to ip-10-0-132-19.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd to ip-10-0-137-200.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp to ip-10-0-137-200.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Successfully assigned openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx to ip-10-0-132-19.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln Successfully assigned openshift-ingress/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln to ip-10-0-132-19.us-east-2.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd to ip-10-0-137-200.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 to ip-10-0-132-19.us-east-2.compute.internal 16m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z to ip-10-0-130-164.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr to ip-10-0-130-164.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Successfully assigned openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s to ip-10-0-132-19.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 to ip-10-0-137-200.us-east-2.compute.internal 28m Warning FailedScheduling pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Warning FailedScheduling pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Normal Scheduled pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk Successfully assigned openshift-ingress/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk to ip-10-0-132-19.us-east-2.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-hcvdj to ip-10-0-130-164.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv to ip-10-0-130-164.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 to ip-10-0-137-200.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 to ip-10-0-137-200.us-east-2.compute.internal 16m Normal Scheduled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Successfully assigned openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz to ip-10-0-137-200.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Successfully assigned openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 to ip-10-0-130-164.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx to ip-10-0-130-164.us-east-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-fpplz to ip-10-0-132-19.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b to ip-10-0-130-164.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Successfully assigned openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw to ip-10-0-137-200.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l to ip-10-0-130-164.us-east-2.compute.internal 8m41s Normal Scheduled pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Successfully assigned openshift-ingress/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c to ip-10-0-132-19.us-east-2.compute.internal 5m23s Normal Scheduled pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Successfully assigned openshift-ingress/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv to ip-10-0-132-19.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w Successfully assigned openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w to ip-10-0-137-200.us-east-2.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-cllrc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-cllrc to ip-10-0-130-164.us-east-2.compute.internal 22m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv to ip-10-0-130-164.us-east-2.compute.internal 19m Normal Scheduled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 Successfully assigned openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 to ip-10-0-132-19.us-east-2.compute.internal 19m Warning FailedScheduling pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 30m Normal Scheduled pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w Successfully assigned openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w to ip-10-0-132-19.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 to ip-10-0-137-200.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 to ip-10-0-132-19.us-east-2.compute.internal 19m Normal Scheduled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Successfully assigned openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk to ip-10-0-130-164.us-east-2.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-chjvh to ip-10-0-132-19.us-east-2.compute.internal 19m Warning FailedScheduling pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Normal Scheduled pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Successfully assigned openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 to ip-10-0-130-164.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 Successfully assigned openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 to ip-10-0-132-19.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq Successfully assigned openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq to ip-10-0-132-19.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Successfully assigned openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 to ip-10-0-130-164.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t to ip-10-0-132-19.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 to ip-10-0-132-19.us-east-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg to ip-10-0-137-200.us-east-2.compute.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk to ip-10-0-137-200.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt to ip-10-0-137-200.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 26m Normal Scheduled pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Successfully assigned openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm to ip-10-0-137-200.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Successfully assigned openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-srmqr to ip-10-0-132-19.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 to ip-10-0-132-19.us-east-2.compute.internal 23m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 to ip-10-0-137-200.us-east-2.compute.internal 24m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 to ip-10-0-132-19.us-east-2.compute.internal 19m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Warning FailedScheduling pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Successfully assigned openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 to ip-10-0-130-164.us-east-2.compute.internal 21m Warning FailedScheduling pod/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 51m Normal Scheduled pod/kube-auth-proxy-678bf485d7-kpxzm Successfully assigned openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm to ip-10-0-137-200.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Successfully assigned openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm to ip-10-0-137-200.us-east-2.compute.internal 28m Warning FailedScheduling pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Warning FailedScheduling pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 18m Warning FailedScheduling pod/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj to ip-10-0-137-200.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw Successfully assigned openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw to ip-10-0-132-19.us-east-2.compute.internal 51m Normal Scheduled pod/kube-auth-proxy-678bf485d7-bnc5s Successfully assigned openshift-ingress/kube-auth-proxy-678bf485d7-bnc5s to ip-10-0-132-19.us-east-2.compute.internal 18m Normal Scheduled pod/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 Successfully assigned openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 to ip-10-0-137-200.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz to ip-10-0-130-164.us-east-2.compute.internal 25m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd to ip-10-0-132-19.us-east-2.compute.internal 23m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 to ip-10-0-137-200.us-east-2.compute.internal 40m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-kk2r9 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-kk2r9 to ip-10-0-132-19.us-east-2.compute.internal 36m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj to ip-10-0-137-200.us-east-2.compute.internal 38m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-5j4cl to ip-10-0-130-164.us-east-2.compute.internal 22m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Normal Scheduled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Successfully assigned openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 to ip-10-0-130-164.us-east-2.compute.internal 17m Normal Scheduled pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Successfully assigned openshift-ingress/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc to ip-10-0-130-164.us-east-2.compute.internal 50m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-2ltm4 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4 to ip-10-0-137-200.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 Successfully assigned openshift-ingress/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 to ip-10-0-132-19.us-east-2.compute.internal 22m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Warning FailedScheduling pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 51m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-4fwqk Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk to ip-10-0-137-200.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Successfully assigned openshift-ingress/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p to ip-10-0-130-164.us-east-2.compute.internal 33m Warning FailedScheduling pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Warning FailedScheduling pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 63m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-ddbc459d5 from 0 to 1 62m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 62m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 62m Normal SuccessfulCreate replicaset/router-default-ddbc459d5 Created pod: router-default-ddbc459d5-jmrh9 62m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 57m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 57m Warning FailedMount pod/router-default-ddbc459d5-jmrh9 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 57m Warning FailedMount pod/router-default-ddbc459d5-jmrh9 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 57m Warning FailedMount pod/router-default-ddbc459d5-jmrh9 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 57m Warning FailedMount pod/router-default-ddbc459d5-jmrh9 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 56m Normal AddedInterface pod/router-default-ddbc459d5-jmrh9 Add eth0 [10.132.0.13/23] from ovn-kubernetes 56m Normal Pulled pod/router-default-ddbc459d5-jmrh9 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 56m Normal Started pod/router-default-ddbc459d5-jmrh9 Container started 56m Normal Created pod/router-default-ddbc459d5-jmrh9 Container created 51m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-678bf485d7 from 0 to 2 51m Normal AddedInterface pod/kube-auth-proxy-678bf485d7-kpxzm Add eth0 [10.133.0.25/23] from ovn-kubernetes 51m Normal Pulling pod/kube-auth-proxy-678bf485d7-kpxzm Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 51m Normal SuccessfulCreate replicaset/kube-auth-proxy-678bf485d7 Created pod: kube-auth-proxy-678bf485d7-kpxzm 51m Normal SuccessfulCreate replicaset/kube-auth-proxy-678bf485d7 Created pod: kube-auth-proxy-678bf485d7-bnc5s 51m Warning FailedMount pod/kube-auth-proxy-678bf485d7-bnc5s MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found 51m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 51m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-4fwqk 51m Normal Pulling pod/kube-auth-proxy-678bf485d7-bnc5s Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 51m Normal AddedInterface pod/kube-auth-proxy-678bf485d7-bnc5s Add eth0 [10.134.0.25/23] from ovn-kubernetes 51m Normal Pulled pod/kube-auth-proxy-678bf485d7-kpxzm Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.249s (2.249s including waiting). Image size: 188904677 bytes. 51m Normal Started pod/kube-auth-proxy-678bf485d7-kpxzm Container started 51m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-4fwqk Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 51m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-4fwqk Add eth0 [10.133.0.27/23] from ovn-kubernetes 51m Normal Created pod/kube-auth-proxy-678bf485d7-kpxzm Container created 51m Normal Created pod/kube-auth-proxy-678bf485d7-bnc5s Container created 51m Normal Started pod/kube-auth-proxy-678bf485d7-bnc5s Container started 51m Normal Pulled pod/kube-auth-proxy-678bf485d7-bnc5s Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.309s (2.309s including waiting). Image size: 188904677 bytes. 51m Normal Created pod/istiod-openshift-gateway-7876bfcb74-4fwqk Container created 51m Normal Started pod/istiod-openshift-gateway-7876bfcb74-4fwqk Container started 51m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-4fwqk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.263s (3.263s including waiting). Image size: 185476477 bytes. 51m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-4fwqk Readiness probe failed: HTTP probe failed with statuscode: 503 51m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-4fwqk Readiness probe error: HTTP probe failed with statuscode: 503... 51m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-68d996c557 Created pod: data-science-gateway-data-science-gateway-class-68d996c5577pfxb 51m Normal Pulling pod/data-science-gateway-data-science-gateway-class-68d996c5577pfxb Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 51m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-68d996c557 from 0 to 1 51m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-68d996c5577pfxb Add eth0 [10.132.0.28/23] from ovn-kubernetes 51m Normal Started pod/data-science-gateway-data-science-gateway-class-68d996c5577pfxb Container started 51m Normal Created pod/data-science-gateway-data-science-gateway-class-68d996c5577pfxb Container created 51m Normal Pulled pod/data-science-gateway-data-science-gateway-class-68d996c5577pfxb Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.938s (2.938s including waiting). Image size: 183555333 bytes. 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-2ltm4 Add eth0 [10.133.0.30/23] from ovn-kubernetes 50m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-2ltm4 Container created 50m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 50m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-2ltm4 50m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-2ltm4 Container started 50m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-2ltm4 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 50m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-4fwqk Stopping container discovery 50m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 50m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-4fwqk 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-cllrc 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-855f9574cf from 0 to 1 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-cllrc Add eth0 [10.132.0.34/23] from ovn-kubernetes 46m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-cllrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-cllrc Container started 46m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-cllrc Container created 46m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-cllrc Startup probe error: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-cllrc Startup probe failed: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused 45m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-855f9574cf from 1 to 4 45m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-bt7bg 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-fpplz 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-rc4rt 45m Normal Pulling pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Add eth0 [10.134.0.34/23] from ovn-kubernetes 45m Normal Pulling pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg Add eth0 [10.133.0.53/23] from ovn-kubernetes 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt Add eth0 [10.133.0.52/23] from ovn-kubernetes 45m Normal Pulling pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 44m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-gblqx 44m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.797s (3.797s including waiting). Image size: 183555333 bytes. 44m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-svc7x 44m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt Container created 44m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.57s (3.57s including waiting). Image size: 183555333 bytes. 44m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Container created 44m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Container started 44m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 44m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 44m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 44m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.248s (3.248s including waiting). Image size: 183555333 bytes. 44m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg Container started 44m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg Container created 44m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt Container started 44m Normal AddedInterface pod/payload-processing-5f78687d5b-gblqx Add eth0 [10.134.0.35/23] from ovn-kubernetes 44m Normal Pulling pod/payload-pre-processing-7b98bb678c-svc7x Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 44m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-svc7x Add eth0 [10.134.0.36/23] from ovn-kubernetes 44m Normal Pulling pod/payload-processing-5f78687d5b-gblqx Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 44m Normal Pulled pod/payload-processing-5f78687d5b-gblqx Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.308s (3.308s including waiting). Image size: 125353965 bytes. 44m Normal Created pod/payload-processing-5f78687d5b-gblqx Container created 44m Normal Pulled pod/payload-pre-processing-7b98bb678c-svc7x Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.17s (3.17s including waiting). Image size: 125353965 bytes. 44m Normal Created pod/payload-pre-processing-7b98bb678c-svc7x Container created 44m Normal Started pod/payload-pre-processing-7b98bb678c-svc7x Container started 44m Normal Started pod/payload-processing-5f78687d5b-gblqx Container started 44m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 44m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Startup probe error: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused... 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Startup probe failed: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 44m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-chjvh 44m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-855f9574cf from 4 to 8 44m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-pthrg 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-lm7vd 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-hcvdj 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Add eth0 [10.133.0.54/23] from ovn-kubernetes 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Add eth0 [10.134.0.38/23] from ovn-kubernetes 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Add eth0 [10.134.0.37/23] from ovn-kubernetes 44m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Container created 44m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Container started 44m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Container created 44m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Container started 44m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Container created 44m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Container started 44m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Container started 44m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Container created 44m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Add eth0 [10.132.0.40/23] from ovn-kubernetes 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 44m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused... 44m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused 44m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 44m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 44m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled up replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 0 to 1 44m Normal AddedLabel gateway/e2e-worker-w4-27b5d5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-27b5d5 44m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l 44m Normal AddedLabel gateway/e2e-worker-w6-bce71f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-bce71f 44m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l 44m Normal AddedLabel gateway/e2e-worker-w5-2d5d53 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-2d5d53 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled up replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 0 to 1 44m Normal AddedInterface pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Add eth0 [10.134.0.40/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj 44m Normal ScalingReplicaSet deployment/e2e-worker-w3-17e904-openshift-default Scaled up replica set e2e-worker-w3-17e904-openshift-default-7cfdf6f655 from 0 to 1 44m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled up replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-worker-w3-17e904-openshift-default-7cfdf6f655 Created pod: e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 44m Normal AddedLabel gateway/e2e-worker-w1-cfa4bb Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-cfa4bb 44m Normal SuccessfulCreate replicaset/e2e-worker-w4-27b5d5-openshift-default-66b797556c Created pod: e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b 44m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx 44m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled up replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 0 to 1 44m Normal AddedLabel gateway/e2e-worker-w3-17e904 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-17e904 44m Normal AddedLabel gateway/e2e-worker-w2-39b867 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-39b867 44m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl 44m Normal AddedLabel gateway/e2e-worker-w0-ad23c7 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-ad23c7 44m Normal ScalingReplicaSet deployment/e2e-worker-w6-bce71f-openshift-default Scaled up replica set e2e-worker-w6-bce71f-openshift-default-5d99f47777 from 0 to 1 44m Normal ScalingReplicaSet deployment/e2e-worker-w4-27b5d5-openshift-default Scaled up replica set e2e-worker-w4-27b5d5-openshift-default-66b797556c from 0 to 1 44m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Add eth0 [10.134.0.39/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj Add eth0 [10.132.0.42/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Add eth0 [10.132.0.43/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Add eth0 [10.132.0.41/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Add eth0 [10.134.0.42/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Add eth0 [10.134.0.41/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff Created pod: payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-27b5d5 Scaled up replica set payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff Error creating: pods "payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-27b5d5: serviceaccount "payload-processing-e2e-worker-w4-27b5d5" not found 44m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-27b5d5-7bffddc884 Error creating: pods "payload-processing-e2e-worker-w4-27b5d5-7bffddc884-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-27b5d5: serviceaccount "payload-processing-e2e-worker-w4-27b5d5" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-27b5d5-7bffddc884 Created pod: payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-27b5d5 Scaled up replica set payload-processing-e2e-worker-w4-27b5d5-7bffddc884 from 0 to 1 43m Normal Pulling pod/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-17e904 Scaled up replica set payload-processing-e2e-worker-w3-17e904-c6bd6d9dd from 0 to 1 43m Normal Pulling pod/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal AddedInterface pod/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 Add eth0 [10.133.0.60/23] from ovn-kubernetes 43m Normal Pulling pod/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd Error creating: pods "payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-17e904: serviceaccount "payload-processing-e2e-worker-w3-17e904" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd Created pod: payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd Add eth0 [10.133.0.59/23] from ovn-kubernetes 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7 Error creating: pods "payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-17e904: serviceaccount "payload-processing-e2e-worker-w3-17e904" not found 43m Normal Pulling pod/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7 Created pod: payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-17e904 Scaled up replica set payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7 from 0 to 1 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb Add eth0 [10.133.0.56/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h Add eth0 [10.133.0.57/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc Created pod: payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b Error creating: pods "payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-2d5d53: serviceaccount "payload-processing-e2e-worker-w5-2d5d53" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-2d5d53 Scaled up replica set payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff Created pod: payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-bce71f Scaled up replica set payload-processing-e2e-worker-w6-bce71f-7df86d777d from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff Error creating: pods "payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-2d5d53: serviceaccount "payload-processing-e2e-worker-w5-2d5d53" not found 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc Error creating: pods "payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-bce71f: serviceaccount "payload-processing-e2e-worker-w6-bce71f" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-bce71f Scaled up replica set payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b Created pod: payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d Add eth0 [10.133.0.62/23] from ovn-kubernetes 43m Normal Pulling pod/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-2d5d53 Scaled up replica set payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b from 0 to 1 43m Normal AddedInterface pod/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 Add eth0 [10.133.0.61/23] from ovn-kubernetes 43m Normal Pulling pod/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567 Error creating: pods "payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb: serviceaccount "payload-processing-e2e-worker-w1-cfa4bb" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4 Error creating: pods "payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb: serviceaccount "payload-processing-e2e-worker-w1-cfa4bb" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-bce71f-7df86d777d Error creating: pods "payload-processing-e2e-worker-w6-bce71f-7df86d777d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-bce71f: serviceaccount "payload-processing-e2e-worker-w6-bce71f" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567 Created pod: payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-cfa4bb Scaled up replica set payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4 Created pod: payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-cfa4bb Scaled up replica set payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4 from 0 to 1 43m Normal AddedInterface pod/payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 Add eth0 [10.134.0.43/23] from ovn-kubernetes 43m Normal Pulling pod/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-bce71f-7df86d777d Created pod: payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 43m Normal Pulling pod/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh Add eth0 [10.133.0.64/23] from ovn-kubernetes 43m Normal Pulling pod/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-ad23c7 Scaled up replica set payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7 from 0 to 1 43m Normal Started pod/payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 Container started 43m Normal Created pod/payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 Container created 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7 Created pod: payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7 Error creating: pods "payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ad23c7: serviceaccount "payload-processing-e2e-worker-w0-ad23c7" not found 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c Add eth0 [10.133.0.65/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq Add eth0 [10.133.0.63/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-ad23c7 Scaled up replica set payload-processing-e2e-worker-w0-ad23c7-78895666b6 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-ad23c7-78895666b6 Created pod: payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-ad23c7-78895666b6 Error creating: pods "payload-processing-e2e-worker-w0-ad23c7-78895666b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ad23c7: serviceaccount "payload-processing-e2e-worker-w0-ad23c7" not found 43m Normal Pulled pod/payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-39b867-6bb6768bb4 Created pod: payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 43m Normal AddedInterface pod/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 Add eth0 [10.133.0.67/23] from ovn-kubernetes 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-39b867-6bb6768bb4 Error creating: pods "payload-processing-e2e-worker-w2-39b867-6bb6768bb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-39b867: serviceaccount "payload-processing-e2e-worker-w2-39b867" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-39b867 Scaled up replica set payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9 from 0 to 1 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 Add eth0 [10.134.0.44/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 Container created 43m Normal Pulling pod/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-39b867 Scaled up replica set payload-processing-e2e-worker-w2-39b867-6bb6768bb4 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9 Error creating: pods "payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-39b867: serviceaccount "payload-processing-e2e-worker-w2-39b867" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9 Created pod: payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 43m Normal AddedInterface pod/payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 Add eth0 [10.134.0.45/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 Container started 43m Normal Pulled pod/payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 Container created 43m Normal Pulling pod/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 Add eth0 [10.132.0.44/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.196s (2.196s including waiting). Image size: 125353965 bytes. 43m Normal Started pod/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 Container created 43m Normal Started pod/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb Container started 43m Normal Pulled pod/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 9.281s (9.281s including waiting). Image size: 125353965 bytes. 43m Normal Created pod/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb Container created 43m Normal Started pod/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h Container started 43m Normal Created pod/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h Container created 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 9.652s (9.652s including waiting). Image size: 125353965 bytes. 43m Normal Pulled pod/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 8.738s (8.738s including waiting). Image size: 125353965 bytes. 43m Normal Started pod/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 Container started 43m Normal Started pod/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 Container started 43m Normal Started pod/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd Container started 43m Normal Created pod/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 Container created 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 9.294s (9.294s including waiting). Image size: 125353965 bytes. 43m Normal Pulled pod/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 9.578s (9.578s including waiting). Image size: 125353965 bytes. 43m Normal Created pod/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 Container created 43m Normal Created pod/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd Container created 43m Normal Created pod/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh Container created 43m Normal Started pod/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d Container started 43m Normal Started pod/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq Container started 43m Normal Started pod/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh Container started 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 8.428s (8.428s including waiting). Image size: 125353965 bytes. 43m Normal Started pod/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c Container started 43m Normal Pulled pod/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 8.821s (8.822s including waiting). Image size: 125353965 bytes. 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 8.708s (8.708s including waiting). Image size: 125353965 bytes. 43m Normal Created pod/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d Container created 43m Normal Pulled pod/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 7.811s (7.811s including waiting). Image size: 125353965 bytes. 43m Normal Created pod/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c Container created 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 9.071s (9.071s including waiting). Image size: 125353965 bytes. 43m Normal Created pod/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 Container created 43m Normal Started pod/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 Container started 43m Normal Created pod/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq Container created 43m Warning ProbeError pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Readiness probe error: Get "http://10.134.0.40:15021/healthz/ready": read tcp 10.134.0.2:58544->10.134.0.40:15021: read: connection reset by peer... 43m Warning Unhealthy pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Readiness probe failed: Get "http://10.134.0.40:15021/healthz/ready": read tcp 10.134.0.2:58544->10.134.0.40:15021: read: connection reset by peer 43m Normal Pulled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Container started 43m Normal Created pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Container created 43m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Container created 43m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Container started 43m Warning ProbeError pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 43m Normal Pulled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Container created 43m Normal Started pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Container started 43m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj Container started 43m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj Container created 43m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Container started 43m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Container created 43m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Container started 43m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Container created 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused 43m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Container started 43m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Container created 43m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning ProbeError pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Startup probe failed: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused 43m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Startup probe error: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj Startup probe failed: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused 43m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj Startup probe error: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-27b5d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-27b5d5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-bce71f-openshift-default Scaled up replica set e2e-worker-w6-bce71f-openshift-default-5d99f47777 from 1 to 2 42m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled up replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 1 to 2 42m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn 42m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Container created 42m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t Add eth0 [10.134.0.52/23] from ovn-kubernetes 42m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t Container started 42m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t Container created 42m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Container started 42m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Add eth0 [10.132.0.53/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused... 42m Normal ScalingReplicaSet deployment/e2e-worker-w4-27b5d5-openshift-default Scaled up replica set e2e-worker-w4-27b5d5-openshift-default-66b797556c from 1 to 2 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-27b5d5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-worker-w4-27b5d5-openshift-default-66b797556c Created pod: e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 42m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq 42m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Container created 42m Normal AddedInterface pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Add eth0 [10.132.0.55/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled up replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 1 to 2 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal Created pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Container created 42m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Add eth0 [10.134.0.55/23] from ovn-kubernetes 42m Normal Started pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Container started 42m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Container started 42m Warning ProbeError pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 42m Normal SuccessfulCreate replicaset/e2e-worker-w3-17e904-openshift-default-7cfdf6f655 Created pod: e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb 42m Normal ScalingReplicaSet deployment/e2e-worker-w3-17e904-openshift-default Scaled up replica set e2e-worker-w3-17e904-openshift-default-7cfdf6f655 from 1 to 2 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled up replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 1 to 2 42m Normal Started pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z Container started 42m Normal Pulled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z Container created 42m Normal AddedInterface pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z Add eth0 [10.132.0.58/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Add eth0 [10.134.0.58/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Container created 42m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Container started 42m Warning Unhealthy pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-bce71f-openshift-default Scaled up replica set e2e-worker-w6-bce71f-openshift-default-5d99f47777 from 2 to 4 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 42m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled up replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 2 to 3 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 42m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 42m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Add eth0 [10.132.0.61/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 Add eth0 [10.134.0.62/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 Container created 42m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 Container started 42m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled up replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 1 to 2 42m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Container created 42m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 Add eth0 [10.133.0.72/23] from ovn-kubernetes 42m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Container started 42m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 Container started 42m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 Container created 42m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Add eth0 [10.133.0.71/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Container created 42m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Container started 41m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled up replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 3 to 4 41m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m 41m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Add eth0 [10.134.0.65/23] from ovn-kubernetes 41m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Container started 41m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Container created 41m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 41m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Readiness probe error: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Readiness probe failed: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-27b5d5-openshift-default Scaled up replica set e2e-worker-w4-27b5d5-openshift-default-66b797556c from 2 to 4 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-27b5d5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-worker-w4-27b5d5-openshift-default-66b797556c Created pod: e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr 41m Normal SuccessfulCreate replicaset/e2e-worker-w4-27b5d5-openshift-default-66b797556c Created pod: e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 41m Normal Started pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr Container started 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled up replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 2 to 3 41m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Container created 41m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Container started 41m Normal Started pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Container started 41m Normal Created pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Container created 41m Normal Created pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr Container created 41m Normal Pulled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr Add eth0 [10.134.0.68/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Add eth0 [10.133.0.75/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf 41m Normal AddedInterface pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Add eth0 [10.133.0.74/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf (combined from similar events): Created pod: maas-default-gateway-openshift-default-855f9574cf-wz7wl 41m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-v9tpn 41m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-855f9574cf from 8 to 10 41m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-wz7wl Add eth0 [10.134.0.69/23] from ovn-kubernetes 41m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-v9tpn Add eth0 [10.132.0.66/23] from ovn-kubernetes 41m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-v9tpn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-v9tpn Container created 41m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-wz7wl Container started 41m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-wz7wl Container created 41m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-v9tpn Container started 41m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-wz7wl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning ProbeError pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-v9tpn Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 41m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-v9tpn Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 41m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-wz7wl Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-wz7wl Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 41m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled up replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 4 to 6 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-worker-w3-17e904-openshift-default Scaled up replica set e2e-worker-w3-17e904-openshift-default-7cfdf6f655 from 2 to 3 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-27b5d5-openshift-default Scaled up replica set e2e-worker-w4-27b5d5-openshift-default-66b797556c from 4 to 5 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-27b5d5-openshift-default New size: 5; reason: 41m Normal SuccessfulCreate replicaset/e2e-worker-w4-27b5d5-openshift-default-66b797556c Created pod: e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-bce71f-openshift-default Scaled up replica set e2e-worker-w6-bce71f-openshift-default-5d99f47777 from 4 to 8 41m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv 41m Normal SuccessfulCreate replicaset/e2e-worker-w3-17e904-openshift-default-7cfdf6f655 Created pod: e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx 41m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b 41m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw 41m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b Add eth0 [10.132.0.70/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 41m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw Container created 41m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw Container started 41m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Container started 41m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Container created 41m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Container started 41m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b Container started 41m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b Container created 41m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Container created 41m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr Container created 41m Normal Started pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr Container started 41m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Add eth0 [10.134.0.72/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw Add eth0 [10.134.0.71/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Add eth0 [10.133.0.76/23] from ovn-kubernetes 41m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Container started 41m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Container created 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled up replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 2 to 4 41m Normal Started pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l Container started 41m Normal Created pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l Container created 41m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Container started 41m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Container created 41m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Add eth0 [10.132.0.68/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 41m Normal Pulled pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Add eth0 [10.133.0.78/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Add eth0 [10.133.0.80/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr Add eth0 [10.133.0.77/23] from ovn-kubernetes 41m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx Container started 41m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx Container created 41m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx Add eth0 [10.132.0.69/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 41m Normal AddedInterface pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l Add eth0 [10.133.0.79/23] from ovn-kubernetes 41m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Container created 41m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Container started 41m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 41m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Container started 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 5; reason: 41m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Add eth0 [10.134.0.76/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Add eth0 [10.132.0.72/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz 41m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Container created 41m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Add eth0 [10.133.0.81/23] from ovn-kubernetes 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled up replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 3 to 4 41m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Container started 41m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Add eth0 [10.134.0.75/23] from ovn-kubernetes 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 41m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Container started 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled up replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 4 to 5 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled up replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 2 to 4 41m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Container created 41m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd 41m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Container started 41m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr 41m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Container created 41m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Container created 41m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx 40m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled up replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 6 to 8 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r 40m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw 40m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Container created 40m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Add eth0 [10.132.0.75/23] from ovn-kubernetes 40m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Container started 40m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Container started 40m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Add eth0 [10.134.0.77/23] from ovn-kubernetes 40m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Container created 40m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 40m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-kk2r9 40m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 40m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-kk2r9 Add eth0 [10.134.0.81/23] from ovn-kubernetes 40m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-kk2r9 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 40m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-kk2r9 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.385s (3.385s including waiting). Image size: 185476477 bytes. 40m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-kk2r9 Container started 40m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-kk2r9 Container created 40m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Container created 40m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Container started 40m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Container created 40m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp 40m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk 40m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Container started 40m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled up replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 4 to 6 40m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk 40m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Add eth0 [10.132.0.79/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled up replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 4 to 8 40m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Add eth0 [10.133.0.82/23] from ovn-kubernetes 40m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Container created 40m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Container started 40m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Container created 40m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Add eth0 [10.133.0.83/23] from ovn-kubernetes 40m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Add eth0 [10.132.0.81/23] from ovn-kubernetes 40m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Add eth0 [10.133.0.84/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Container started 40m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Container created 40m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Container created 40m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Add eth0 [10.132.0.80/23] from ovn-kubernetes 40m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Container started 40m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Container started 40m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc (combined from similar events): Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Created pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled up replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 8 to 10 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 40m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Add eth0 [10.134.0.84/23] from ovn-kubernetes 40m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Container started 40m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f (combined from similar events): Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj 40m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Add eth0 [10.133.0.87/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv 40m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Add eth0 [10.132.0.84/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled up replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 8 to 9 40m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 40m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr 40m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled up replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 5 to 10 40m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Add eth0 [10.134.0.83/23] from ovn-kubernetes 40m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Add eth0 [10.132.0.83/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Created pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 40m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Container created 40m Normal AddedInterface pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Add eth0 [10.133.0.85/23] from ovn-kubernetes 40m Normal Started pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Container started 40m Normal Created pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Container created 40m Normal AddedInterface pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Add eth0 [10.133.0.86/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Container created 40m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Container started 40m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Container created 40m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Container created 40m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Container started 40m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Container started 40m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Container started 40m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Container created 40m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Container started 40m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Container created 40m Normal Started pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Container started 40m Normal Created pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Container created 40m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Add eth0 [10.134.0.85/23] from ovn-kubernetes 39m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 39m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Container created 39m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Add eth0 [10.132.0.89/23] from ovn-kubernetes 39m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Container created 39m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Container started 39m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Container started 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 39m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Container started 39m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Add eth0 [10.134.0.87/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g 39m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Container created 39m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s 39m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Add eth0 [10.133.0.88/23] from ovn-kubernetes 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled up replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 6 to 9 39m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs 39m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Readiness probe error: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Readiness probe failed: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Readiness probe error: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Readiness probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Readiness probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Readiness probe failed: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Readiness probe failed: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 39m Normal SuccessfulCreate replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 (combined from similar events): Created pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled up replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 9 to 10 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled up replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 9 to 10 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 39m Normal SuccessfulCreate replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f (combined from similar events): Created pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc 39m Normal Pulled pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Add eth0 [10.133.0.89/23] from ovn-kubernetes 39m Normal AddedInterface pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Add eth0 [10.132.0.93/23] from ovn-kubernetes 39m Normal Started pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Container started 39m Normal Created pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Container created 39m Normal Started pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Container started 39m Normal Created pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Container created 39m Normal Pulled pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Readiness probe error: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Readiness probe error: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Readiness probe failed: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal ScalingReplicaSet deployment/e2e-ait-250fb057-openshift-default Scaled up replica set e2e-ait-250fb057-openshift-default-7c6cb4fd67 from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-ait-250fb057-openshift-default-7c6cb4fd67 Created pod: e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh 38m Normal AddedLabel gateway/e2e-ait-250fb057 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-250fb057 38m Normal AddedInterface pod/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh Add eth0 [10.132.0.100/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-migrate-e308e736-gw-openshift-default Scaled up replica set e2e-migrate-e308e736-gw-openshift-default-99b946c6f from 0 to 1 38m Normal AddedLabel gateway/e2e-migrate-e308e736-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-e308e736-gw 38m Normal SuccessfulCreate replicaset/e2e-migrate-e308e736-gw-openshift-default-99b946c6f Created pod: e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-250fb057-7598f9b64 Created pod: payload-processing-e2e-ait-250fb057-7598f9b64-d96bp 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-250fb057-9b44cc67 Error creating: pods "payload-pre-processing-e2e-ait-250fb057-9b44cc67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-250fb057: serviceaccount "payload-processing-e2e-ait-250fb057" not found 38m Normal AddedInterface pod/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 Add eth0 [10.132.0.101/23] from ovn-kubernetes 38m Warning FailedCreate replicaset/payload-processing-e2e-ait-250fb057-7598f9b64 Error creating: pods "payload-processing-e2e-ait-250fb057-7598f9b64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-250fb057: serviceaccount "payload-processing-e2e-ait-250fb057" not found 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-250fb057 Scaled up replica set payload-processing-e2e-ait-250fb057-7598f9b64 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-250fb057 Scaled up replica set payload-pre-processing-e2e-ait-250fb057-9b44cc67 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-250fb057-9b44cc67 Created pod: payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl 38m Normal Pulled pod/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl Container started 38m Normal Created pod/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl Container created 38m Normal Pulled pod/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl Add eth0 [10.133.0.92/23] from ovn-kubernetes 38m Normal Created pod/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp Container created 38m Normal Started pod/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp Container started 38m Normal AddedInterface pod/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp Add eth0 [10.133.0.93/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-e308e736 Scaled up replica set payload-processing-e2e-migrate-e308e736-7596fb59d6 from 0 to 1 38m Warning FailedCreate replicaset/payload-processing-e2e-migrate-e308e736-7596fb59d6 Error creating: pods "payload-processing-e2e-migrate-e308e736-7596fb59d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-e308e736: serviceaccount "payload-processing-e2e-migrate-e308e736" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-e308e736 Scaled up replica set payload-pre-processing-e2e-migrate-e308e736-856845bc79 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-e308e736-7596fb59d6 Created pod: payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-e308e736-856845bc79 Created pod: payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-e308e736-856845bc79 Error creating: pods "payload-pre-processing-e2e-migrate-e308e736-856845bc79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-e308e736: serviceaccount "payload-processing-e2e-migrate-e308e736" not found 38m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Readiness probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk Add eth0 [10.133.0.94/23] from ovn-kubernetes 38m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Readiness probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-5j4cl 38m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Readiness probe error: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Readiness probe failed: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Readiness probe error: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal AddedInterface pod/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt Add eth0 [10.133.0.95/23] from ovn-kubernetes 38m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Readiness probe failed: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Readiness probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Readiness probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Pulled pod/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Readiness probe failed: Get "http://10.133.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Readiness probe failed: Get "http://10.134.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Readiness probe error: Get "http://10.134.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Pulled pod/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Readiness probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Readiness probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Readiness probe error: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Readiness probe error: Get "http://10.133.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Readiness probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Readiness probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Readiness probe failed: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Readiness probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Readiness probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Readiness probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Readiness probe error: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Readiness probe error: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Readiness probe failed: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Readiness probe failed: Get "http://10.132.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Readiness probe failed: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Readiness probe failed: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Readiness probe error: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Readiness probe error: Get "http://10.132.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Readiness probe error: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Readiness probe failed: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Readiness probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Readiness probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Readiness probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Readiness probe failed: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Readiness probe failed: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Readiness probe error: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Readiness probe error: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Created pod/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt Container created 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Readiness probe error: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Started pod/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk Container started 38m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Readiness probe error: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Readiness probe error: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Created pod/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk Container created 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Readiness probe failed: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Readiness probe failed: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Readiness probe failed: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal Started pod/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt Container started 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Readiness probe failed: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Readiness probe error: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 38m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Add eth0 [10.132.0.102/23] from ovn-kubernetes 38m Warning ProbeError pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Readiness probe error: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Readiness probe failed: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Container started 38m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Container created 38m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.843s (2.843s including waiting). Image size: 185476477 bytes. 38m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal SuccessfulCreate replicaset/e2e-ait-7c74535f-openshift-default-f55494c4f Created pod: e2e-ait-7c74535f-openshift-default-f55494c4f-fwpmp 38m Normal AddedInterface pod/e2e-ait-7c74535f-openshift-default-f55494c4f-fwpmp Add eth0 [10.134.0.98/23] from ovn-kubernetes 38m Normal AddedLabel gateway/e2e-ait-7c74535f Added label istio.io/rev=openshift-gateway to gateway e2e-ait-7c74535f 38m Normal ScalingReplicaSet deployment/e2e-ait-7c74535f-openshift-default Scaled up replica set e2e-ait-7c74535f-openshift-default-f55494c4f from 0 to 1 38m Normal Pulled pod/e2e-ait-7c74535f-openshift-default-f55494c4f-fwpmp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-ait-7c74535f-openshift-default-f55494c4f-fwpmp Container started 38m Normal Created pod/e2e-ait-7c74535f-openshift-default-f55494c4f-fwpmp Container created 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-7c74535f Scaled up replica set payload-pre-processing-e2e-ait-7c74535f-7f9967fff9 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-7c74535f Scaled up replica set payload-processing-e2e-ait-7c74535f-74c84655b6 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9 Error creating: pods "payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-7c74535f: serviceaccount "payload-processing-e2e-ait-7c74535f" not found 38m Warning FailedCreate replicaset/payload-processing-e2e-ait-7c74535f-74c84655b6 Error creating: pods "payload-processing-e2e-ait-7c74535f-74c84655b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-7c74535f: serviceaccount "payload-processing-e2e-ait-7c74535f" not found 38m Normal Pulled pod/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc Container created 38m Normal Started pod/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc Container started 38m Normal AddedInterface pod/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc Add eth0 [10.133.0.99/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-7c74535f-74c84655b6 Created pod: payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-250fb057-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9 Created pod: payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-mbjb4 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-250fb057-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal AddedInterface pod/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-mbjb4 Add eth0 [10.134.0.100/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-mbjb4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-mbjb4 Container created 38m Normal Started pod/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-mbjb4 Container started 38m Warning ProbeError pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Readiness probe failed: Get "http://10.132.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Readiness probe error: Get "http://10.132.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Killing pod/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt Stopping container payload-processing 38m Warning ProbeError pod/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh Readiness probe error: Get "http://10.132.0.100:15021/healthz/ready": read tcp 10.132.0.2:36750->10.132.0.100:15021: read: connection reset by peer... 38m Warning Unhealthy pod/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh Readiness probe failed: Get "http://10.132.0.100:15021/healthz/ready": read tcp 10.132.0.2:36750->10.132.0.100:15021: read: connection reset by peer 38m Normal Killing pod/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk Stopping container payload-pre-processing 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-e308e736-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-e308e736-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Pulled pod/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh Container created 38m Normal Pulled pod/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 Container created 38m Normal Started pod/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh Container started 38m Normal Started pod/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 Container started 38m Normal Killing pod/e2e-ait-250fb057-openshift-default-7c6cb4fd67-xzthh Stopping container istio-proxy 38m Normal Pulled pod/e2e-derive-c5ed0670-openshift-default-5f977cdc49-rqrgb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedLabel gateway/e2e-derive-c5ed0670 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-c5ed0670 38m Normal AddedInterface pod/e2e-derive-c5ed0670-openshift-default-5f977cdc49-rqrgb Add eth0 [10.134.0.103/23] from ovn-kubernetes 38m Normal Created pod/e2e-derive-c5ed0670-openshift-default-5f977cdc49-rqrgb Container created 38m Normal Started pod/e2e-derive-c5ed0670-openshift-default-5f977cdc49-rqrgb Container started 38m Normal SuccessfulCreate replicaset/e2e-derive-c5ed0670-openshift-default-5f977cdc49 Created pod: e2e-derive-c5ed0670-openshift-default-5f977cdc49-rqrgb 38m Normal ScalingReplicaSet deployment/e2e-derive-c5ed0670-openshift-default Scaled up replica set e2e-derive-c5ed0670-openshift-default-5f977cdc49 from 0 to 1 37m Normal Killing pod/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-c5ed0670 Scaled up replica set payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75 Created pod: payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75 Error creating: pods "payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c5ed0670: serviceaccount "payload-processing-e2e-derive-c5ed0670" not found 37m Warning Unhealthy pod/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 37m Warning ProbeError pod/e2e-migrate-e308e736-gw-openshift-default-99b946c6f-5w282 Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-c5ed0670 Scaled up replica set payload-processing-e2e-derive-c5ed0670-7df696cc96 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-c5ed0670-7df696cc96 Created pod: payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5 37m Warning FailedCreate replicaset/payload-processing-e2e-derive-c5ed0670-7df696cc96 Error creating: pods "payload-processing-e2e-derive-c5ed0670-7df696cc96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c5ed0670: serviceaccount "payload-processing-e2e-derive-c5ed0670" not found 37m Normal Started pod/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j Container started 37m Normal AddedInterface pod/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j Add eth0 [10.133.0.103/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5 Container started 37m Normal AddedInterface pod/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5 Add eth0 [10.133.0.102/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j Container created 37m Normal Created pod/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5 Container created 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-7c74535f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-7c74535f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 Add eth0 [10.133.0.105/23] from ovn-kubernetes 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-worker-w3-17e904-openshift-default-7cfdf6f655 Created pod: e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj 37m Normal AddedInterface pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 Add eth0 [10.134.0.105/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/e2e-worker-w3-17e904-openshift-default Scaled up replica set e2e-worker-w3-17e904-openshift-default-7cfdf6f655 from 3 to 5 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-worker-w3-17e904-openshift-default-7cfdf6f655 Created pod: e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-bce71f-openshift-default Scaled up replica set e2e-worker-w6-bce71f-openshift-default-5d99f47777 from 8 to 10 37m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 (combined from similar events): Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 37m Normal SuccessfulCreate replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Created pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 37m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 Container started 37m Normal Created pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj Container created 37m Normal Pulled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj Add eth0 [10.134.0.106/23] from ovn-kubernetes 37m Normal Started pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj Container started 37m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 Container created 37m Normal Created pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj Container created 37m Normal Created pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 Container created 37m Normal Pulled pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj Add eth0 [10.132.0.106/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj Container started 37m Normal Started pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 Container started 37m Normal Killing pod/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-ait-7c74535f-7f9967fff9-mbjb4 Stopping container payload-pre-processing 37m Warning ProbeError pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 37m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 37m Normal Killing pod/e2e-ait-7c74535f-openshift-default-f55494c4f-fwpmp Stopping container istio-proxy 37m Warning Unhealthy pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused 37m Warning ProbeError pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-c5ed0670-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-c5ed0670-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Created pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-2405fb-openshift-default Scaled up replica set e2e-shared-a-w0-2405fb-openshift-default-556d785b45 from 0 to 1 37m Normal AddedLabel gateway/e2e-shared-a-w0-2405fb Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-2405fb 37m Normal AddedInterface pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Add eth0 [10.134.0.108/23] from ovn-kubernetes 37m Normal Killing pod/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5 Stopping container payload-processing 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc Created pod: payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc Error creating: pods "payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb: serviceaccount "payload-processing-e2e-shared-a-w0-2405fb" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-2405fb Scaled up replica set payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-2405fb Scaled up replica set payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8 Created pod: payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8 Error creating: pods "payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb: serviceaccount "payload-processing-e2e-shared-a-w0-2405fb" not found 37m Normal Created pod/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 Container created 37m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk Container started 37m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk Container created 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk Add eth0 [10.133.0.106/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 Container started 37m Normal Pulled pod/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 Add eth0 [10.133.0.107/23] from ovn-kubernetes 37m Normal Killing pod/e2e-derive-c5ed0670-openshift-default-5f977cdc49-rqrgb Stopping container istio-proxy 37m Normal AddedLabel gateway/e2e-shared-a-w1-618d93 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-618d93 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-618d93-openshift-default Scaled up replica set e2e-shared-a-w1-618d93-openshift-default-79769f779 from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-618d93-openshift-default-79769f779 Created pod: e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 37m Normal AddedInterface pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 Add eth0 [10.132.0.107/23] from ovn-kubernetes 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6 Created pod: payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6 Error creating: pods "payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-618d93: serviceaccount "payload-processing-e2e-shared-a-w1-618d93" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-618d93 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695 Created pod: payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-618d93: serviceaccount "payload-processing-e2e-shared-a-w1-618d93" not found 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-618d93 Scaled up replica set payload-processing-e2e-shared-a-w1-618d93-68dd6997f6 from 0 to 1 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g Add eth0 [10.133.0.110/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d Add eth0 [10.133.0.109/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d Container created 37m Normal Started pod/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d Container started 37m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g Container created 37m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g Container started 37m Normal AddedLabel gateway/e2e-mt-0d360546 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-0d360546 37m Normal SuccessfulCreate replicaset/e2e-mt-0d360546-openshift-default-5587bbdb7c Created pod: e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z 37m Normal ScalingReplicaSet deployment/e2e-mt-0d360546-openshift-default Scaled up replica set e2e-mt-0d360546-openshift-default-5587bbdb7c from 0 to 1 37m Normal AddedInterface pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Add eth0 [10.134.0.109/23] from ovn-kubernetes 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-0d360546-569bbdf84c Created pod: payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-0d360546-569bbdf84c Error creating: pods "payload-pre-processing-e2e-mt-0d360546-569bbdf84c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-0d360546: serviceaccount "payload-processing-e2e-mt-0d360546" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-mt-0d360546-75f95c9c6 Error creating: pods "payload-processing-e2e-mt-0d360546-75f95c9c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-0d360546: serviceaccount "payload-processing-e2e-mt-0d360546" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-0d360546-75f95c9c6 Created pod: payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-0d360546 Scaled up replica set payload-pre-processing-e2e-mt-0d360546-569bbdf84c from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-0d360546 Scaled up replica set payload-processing-e2e-mt-0d360546-75f95c9c6 from 0 to 1 37m Normal Started pod/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69 Container started 37m Normal Created pod/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69 Container created 37m Normal Created pod/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl Container created 37m Normal AddedInterface pod/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69 Add eth0 [10.133.0.112/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl Add eth0 [10.133.0.113/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl Container started 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-1e7123-openshift-default Scaled up replica set e2e-shared-b-w0-1e7123-openshift-default-796568db5f from 0 to 1 37m Normal AddedLabel gateway/e2e-shared-b-w0-1e7123 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-1e7123 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-1e7123-openshift-default-796568db5f Created pod: e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x 37m Normal AddedInterface pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x Add eth0 [10.132.0.108/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-1e7123 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-1e7123 Scaled up replica set payload-processing-e2e-shared-b-w0-1e7123-c96dc489b from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b Created pod: payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b Error creating: pods "payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123: serviceaccount "payload-processing-e2e-shared-b-w0-1e7123" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f Created pod: payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f Error creating: pods "payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123: serviceaccount "payload-processing-e2e-shared-b-w0-1e7123" not found 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-2405fb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-2405fb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz Add eth0 [10.134.0.110/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s Container created 36m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz Container created 36m Normal Started pod/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s Container started 36m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s Add eth0 [10.133.0.114/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedLabel gateway/e2e-shared-b-w1-e3586f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-e3586f 36m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Readiness probe failed: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e3586f-openshift-default Scaled up replica set e2e-shared-b-w1-e3586f-openshift-default-976787cfc from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e3586f-openshift-default-976787cfc Created pod: e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt 36m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Readiness probe error: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal AddedInterface pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Add eth0 [10.134.0.111/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Container created 36m Normal Started pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Container started 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6 Created pod: payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6 Error creating: pods "payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-e3586f: serviceaccount "payload-processing-e2e-shared-b-w1-e3586f" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-e3586f Scaled up replica set payload-processing-e2e-shared-b-w1-e3586f-77459df5d6 from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-e3586f: serviceaccount "payload-processing-e2e-shared-b-w1-e3586f" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-e3586f Scaled up replica set payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5 Created pod: payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 36m Normal Created pod/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 Container created 36m Normal Started pod/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 Container started 36m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 Container created 36m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 Add eth0 [10.132.0.109/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 Add eth0 [10.134.0.112/23] from ovn-kubernetes 36m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 Container started 36m Normal Pulled pod/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused... 36m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 36m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 36m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-b7qjj 36m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj Add eth0 [10.133.0.118/23] from ovn-kubernetes 36m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj Container started 36m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj Container created 36m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 36m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal AddedLabel gateway/e2e-mt-48b1a6a7 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-48b1a6a7 36m Normal AddedInterface pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Add eth0 [10.134.0.113/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/e2e-mt-48b1a6a7-openshift-default Scaled up replica set e2e-mt-48b1a6a7-openshift-default-78f6598f5c from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-mt-48b1a6a7-openshift-default-78f6598f5c Created pod: e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-2405fb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-2405fb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedCreate replicaset/payload-processing-e2e-mt-48b1a6a7-5547c65796 Error creating: pods "payload-processing-e2e-mt-48b1a6a7-5547c65796-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-48b1a6a7: serviceaccount "payload-processing-e2e-mt-48b1a6a7" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95 Error creating: pods "payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-48b1a6a7: serviceaccount "payload-processing-e2e-mt-48b1a6a7" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-48b1a6a7-5547c65796 Created pod: payload-processing-e2e-mt-48b1a6a7-5547c65796-crcbs 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-48b1a6a7 Scaled up replica set payload-processing-e2e-mt-48b1a6a7-5547c65796 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-48b1a6a7 Scaled up replica set payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95 Created pod: payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-rmdbz 36m Normal Pulled pod/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-rmdbz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-rmdbz Add eth0 [10.132.0.112/23] from ovn-kubernetes 36m Normal Created pod/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-rmdbz Container created 36m Normal Started pod/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-rmdbz Container started 36m Normal AddedInterface pod/payload-processing-e2e-mt-48b1a6a7-5547c65796-crcbs Add eth0 [10.132.0.113/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-mt-48b1a6a7-5547c65796-crcbs Container started 36m Normal Pulled pod/payload-processing-e2e-mt-48b1a6a7-5547c65796-crcbs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-mt-48b1a6a7-5547c65796-crcbs Container created 36m Warning Unhealthy pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Readiness probe failed: Get "http://10.134.0.109:15021/healthz/ready": read tcp 10.134.0.2:35396->10.134.0.109:15021: read: connection reset by peer 36m Normal Created pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Container created 36m Normal Pulled pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Container started 36m Warning ProbeError pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Readiness probe error: Get "http://10.134.0.109:15021/healthz/ready": read tcp 10.134.0.2:35396->10.134.0.109:15021: read: connection reset by peer... 36m Normal Pulled pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x Container started 36m Normal Created pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x Container created 36m Normal Started pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 Container started 36m Normal Created pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 Container created 36m Normal Pulled pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-618d93-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-618d93-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal AddedLabel gateway/e2e-api-a-9a718a Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-9a718a 36m Normal ScalingReplicaSet deployment/e2e-api-a-9a718a-openshift-default Scaled up replica set e2e-api-a-9a718a-openshift-default-698df85f8c from 0 to 1 36m Warning Unhealthy pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Startup probe failed: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused 36m Normal SuccessfulCreate replicaset/e2e-api-a-9a718a-openshift-default-698df85f8c Created pod: e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k 36m Warning ProbeError pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Startup probe error: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused... 36m Normal AddedInterface pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Add eth0 [10.134.0.115/23] from ovn-kubernetes 36m Warning ProbeError pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 36m Normal AddedLabel gateway/e2e-shared-a-w2-b281d5 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-b281d5 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-b281d5-openshift-default Scaled up replica set e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5 Created pod: e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs 36m Normal AddedInterface pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Add eth0 [10.132.0.118/23] from ovn-kubernetes 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-0d360546-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-9a718a Scaled up replica set payload-processing-e2e-api-a-9a718a-6f7445cb4d from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-9a718a-67df85b967 Error creating: pods "payload-pre-processing-e2e-api-a-9a718a-67df85b967-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-9a718a: serviceaccount "payload-processing-e2e-api-a-9a718a" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-9a718a Scaled up replica set payload-pre-processing-e2e-api-a-9a718a-67df85b967 from 0 to 1 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-0d360546-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-9a718a-67df85b967 Created pod: payload-pre-processing-e2e-api-a-9a718a-67df85b967-xbx45 36m Warning FailedCreate replicaset/payload-processing-e2e-api-a-9a718a-6f7445cb4d Error creating: pods "payload-processing-e2e-api-a-9a718a-6f7445cb4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-9a718a: serviceaccount "payload-processing-e2e-api-a-9a718a" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-9a718a-6f7445cb4d Created pod: payload-processing-e2e-api-a-9a718a-6f7445cb4d-h2hxs 36m Normal Pulled pod/payload-pre-processing-e2e-api-a-9a718a-67df85b967-xbx45 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-api-a-9a718a-6f7445cb4d-h2hxs Container created 36m Normal Created pod/payload-pre-processing-e2e-api-a-9a718a-67df85b967-xbx45 Container created 36m Normal AddedInterface pod/payload-processing-e2e-api-a-9a718a-6f7445cb4d-h2hxs Add eth0 [10.132.0.119/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-9a718a-67df85b967-xbx45 Add eth0 [10.134.0.116/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-api-a-9a718a-6f7445cb4d-h2hxs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-processing-e2e-api-a-9a718a-6f7445cb4d-h2hxs Container started 36m Normal Started pod/payload-pre-processing-e2e-api-a-9a718a-67df85b967-xbx45 Container started 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-b281d5 Scaled up replica set payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6 Created pod: payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6 Error creating: pods "payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-b281d5: serviceaccount "payload-processing-e2e-shared-a-w2-b281d5" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c Error creating: pods "payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-b281d5: serviceaccount "payload-processing-e2e-shared-a-w2-b281d5" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-b281d5 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c Created pod: payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h 36m Normal Created pod/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk Container created 36m Normal Pulled pod/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h Container started 36m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h Container created 36m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h Add eth0 [10.132.0.120/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk Container started 36m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk Add eth0 [10.132.0.121/23] from ovn-kubernetes 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-1e7123-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-1e7123-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-0d360546-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-0d360546-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-e3586f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-e3586f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Created pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Container created 35m Normal Started pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Container started 35m Normal Pulled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-618d93-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-618d93-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-1e7123-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-1e7123-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal AddedLabel gateway/e2e-api-b-9b2e27 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-9b2e27 35m Normal ScalingReplicaSet deployment/e2e-api-b-9b2e27-openshift-default Scaled up replica set e2e-api-b-9b2e27-openshift-default-8489bb687c from 0 to 1 35m Normal AddedInterface pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Add eth0 [10.134.0.118/23] from ovn-kubernetes 35m Normal SuccessfulCreate replicaset/e2e-api-b-9b2e27-openshift-default-8489bb687c Created pod: e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-8f3ff4-openshift-default Scaled up replica set e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9 from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9 Created pod: e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n 35m Normal AddedLabel gateway/e2e-shared-b-w2-8f3ff4 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-8f3ff4 35m Normal AddedInterface pod/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n Add eth0 [10.132.0.124/23] from ovn-kubernetes 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-9b2e27-56c4c6f766 Created pod: payload-processing-e2e-api-b-9b2e27-56c4c6f766-fphsc 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-9b2e27 Scaled up replica set payload-processing-e2e-api-b-9b2e27-56c4c6f766 from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-api-b-9b2e27-56c4c6f766 Error creating: pods "payload-processing-e2e-api-b-9b2e27-56c4c6f766-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-9b2e27: serviceaccount "payload-processing-e2e-api-b-9b2e27" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-9b2e27 Scaled up replica set payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9 Created pod: payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-8f9j5 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9 Error creating: pods "payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-9b2e27: serviceaccount "payload-processing-e2e-api-b-9b2e27" not found 35m Normal Pulled pod/payload-processing-e2e-api-b-9b2e27-56c4c6f766-fphsc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-api-b-9b2e27-56c4c6f766-fphsc Add eth0 [10.132.0.125/23] from ovn-kubernetes 35m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-8f9j5 Add eth0 [10.132.0.126/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-api-b-9b2e27-56c4c6f766-fphsc Container started 35m Normal Started pod/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-8f9j5 Container started 35m Normal Created pod/payload-processing-e2e-api-b-9b2e27-56c4c6f766-fphsc Container created 35m Normal Created pod/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-8f9j5 Container created 35m Normal Pulled pod/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-8f9j5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84 Created pod: payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-8f3ff4: serviceaccount "payload-processing-e2e-shared-b-w2-8f3ff4" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-8f3ff4 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59 Created pod: payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84 Error creating: pods "payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-8f3ff4: serviceaccount "payload-processing-e2e-shared-b-w2-8f3ff4" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-8f3ff4 Scaled up replica set payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84 from 0 to 1 35m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp Add eth0 [10.134.0.120/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx Container created 35m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx Container started 35m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp Container created 35m Normal Pulled pod/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp Container started 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx Add eth0 [10.134.0.119/23] from ovn-kubernetes 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5 Created pod: e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5 Created pod: e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5 Created pod: e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-b281d5-openshift-default Scaled up replica set e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5 from 1 to 4 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-b281d5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 35m Normal AddedInterface pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds Add eth0 [10.134.0.121/23] from ovn-kubernetes 35m Normal Started pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h Container started 35m Normal Pulled pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Readiness probe error: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal Created pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds Container created 35m Normal Started pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds Container started 35m Normal AddedInterface pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h Add eth0 [10.133.0.119/23] from ovn-kubernetes 35m Normal Created pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h Container created 35m Normal AddedInterface pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 Add eth0 [10.132.0.127/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 Container created 35m Normal Pulled pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 Container started 35m Warning Unhealthy pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Readiness probe failed: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-e3586f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-e3586f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning Unhealthy pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Readiness probe failed: Get "http://10.132.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Normal Pulled pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Container created 35m Warning ProbeError pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Readiness probe error: Get "http://10.132.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal Started pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Container started 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-9b2e27-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-9b2e27-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Pulled pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning Unhealthy pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 35m Normal Created pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Container created 35m Normal Started pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Container started 35m Warning ProbeError pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds Startup probe failed: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds Startup probe error: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused 35m Warning ProbeError pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-b281d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-b281d5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-9a718a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-9a718a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal ScalingReplicaSet deployment/e2e-api-a-8ceda0-openshift-default Scaled up replica set e2e-api-a-8ceda0-openshift-default-6849d5bcf from 0 to 1 35m Normal AddedLabel gateway/e2e-api-a-8ceda0 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-8ceda0 35m Normal SuccessfulCreate replicaset/e2e-api-a-8ceda0-openshift-default-6849d5bcf Created pod: e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr 34m Normal Killing pod/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69 Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl Stopping container payload-pre-processing 34m Normal AddedInterface pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Add eth0 [10.132.0.130/23] from ovn-kubernetes 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-b281d5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-b281d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal AddedLabel gateway/e2e-api-a-f4012a Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-f4012a 34m Normal ScalingReplicaSet deployment/e2e-api-a-f4012a-openshift-default Scaled up replica set e2e-api-a-f4012a-openshift-default-7c884cc469 from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-api-a-f4012a-openshift-default-7c884cc469 Created pod: e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-8ceda0-54c896b567 Error creating: pods "payload-pre-processing-e2e-api-a-8ceda0-54c896b567-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-8ceda0: serviceaccount "payload-processing-e2e-api-a-8ceda0" not found 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-8ceda0 Scaled up replica set payload-pre-processing-e2e-api-a-8ceda0-54c896b567 from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-api-a-8ceda0-6ff4cc8588 Error creating: pods "payload-processing-e2e-api-a-8ceda0-6ff4cc8588-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-8ceda0: serviceaccount "payload-processing-e2e-api-a-8ceda0" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-8ceda0-6ff4cc8588 Created pod: payload-processing-e2e-api-a-8ceda0-6ff4cc8588-xbn7c 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-8ceda0 Scaled up replica set payload-processing-e2e-api-a-8ceda0-6ff4cc8588 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-8ceda0-54c896b567 Created pod: payload-pre-processing-e2e-api-a-8ceda0-54c896b567-45659 34m Normal AddedInterface pod/payload-processing-e2e-api-a-8ceda0-6ff4cc8588-xbn7c Add eth0 [10.134.0.123/23] from ovn-kubernetes 34m Normal AddedInterface pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Add eth0 [10.134.0.122/23] from ovn-kubernetes 34m Normal Started pod/payload-pre-processing-e2e-api-a-8ceda0-54c896b567-45659 Container started 34m Normal Created pod/payload-pre-processing-e2e-api-a-8ceda0-54c896b567-45659 Container created 34m Normal Pulled pod/payload-pre-processing-e2e-api-a-8ceda0-54c896b567-45659 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-8ceda0-54c896b567-45659 Add eth0 [10.132.0.131/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-api-a-8ceda0-6ff4cc8588-xbn7c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-api-a-8ceda0-6ff4cc8588-xbn7c Container created 34m Normal Started pod/payload-processing-e2e-api-a-8ceda0-6ff4cc8588-xbn7c Container started 34m Normal Killing pod/payload-processing-e2e-api-a-9a718a-6f7445cb4d-h2hxs Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-api-a-9a718a-67df85b967-xbx45 Stopping container payload-pre-processing 34m Warning Unhealthy pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Readiness probe failed: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Warning ProbeError pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Readiness probe error: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Warning FailedCreate replicaset/payload-processing-e2e-api-a-f4012a-78c8dd4c94 Error creating: pods "payload-processing-e2e-api-a-f4012a-78c8dd4c94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f4012a: serviceaccount "payload-processing-e2e-api-a-f4012a" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-f4012a Scaled up replica set payload-processing-e2e-api-a-f4012a-78c8dd4c94 from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-f4012a-b58495cbc Error creating: pods "payload-pre-processing-e2e-api-a-f4012a-b58495cbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f4012a: serviceaccount "payload-processing-e2e-api-a-f4012a" not found 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-f4012a Scaled up replica set payload-pre-processing-e2e-api-a-f4012a-b58495cbc from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-f4012a-78c8dd4c94 Created pod: payload-processing-e2e-api-a-f4012a-78c8dd4c94-sgr5m 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-f4012a-b58495cbc Created pod: payload-pre-processing-e2e-api-a-f4012a-b58495cbc-zdz5w 34m Normal Pulled pod/payload-pre-processing-e2e-api-a-f4012a-b58495cbc-zdz5w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-f4012a-b58495cbc-zdz5w Add eth0 [10.134.0.124/23] from ovn-kubernetes 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-9a718a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal AddedInterface pod/payload-processing-e2e-api-a-f4012a-78c8dd4c94-sgr5m Add eth0 [10.134.0.125/23] from ovn-kubernetes 34m Normal Created pod/payload-processing-e2e-api-a-f4012a-78c8dd4c94-sgr5m Container created 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-9a718a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Pulled pod/payload-processing-e2e-api-a-f4012a-78c8dd4c94-sgr5m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-pre-processing-e2e-api-a-f4012a-b58495cbc-zdz5w Container started 34m Normal Created pod/payload-pre-processing-e2e-api-a-f4012a-b58495cbc-zdz5w Container created 34m Normal Started pod/payload-processing-e2e-api-a-f4012a-78c8dd4c94-sgr5m Container started 34m Normal Created pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Container created 34m Normal Created pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Container created 34m Normal Pulled pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Pulled pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Container started 34m Normal Started pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Container started 34m Normal AddedLabel gateway/e2e-inf-a-33b4d7 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-33b4d7 34m Normal SuccessfulCreate replicaset/e2e-inf-a-33b4d7-openshift-default-5c9994896f Created pod: e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt 34m Normal ScalingReplicaSet deployment/e2e-inf-a-33b4d7-openshift-default Scaled up replica set e2e-inf-a-33b4d7-openshift-default-5c9994896f from 0 to 1 34m Normal AddedInterface pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Add eth0 [10.132.0.133/23] from ovn-kubernetes 34m Normal Killing pod/e2e-mt-0d360546-openshift-default-5587bbdb7c-wnc8z Stopping container istio-proxy 34m Warning Unhealthy pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Startup probe failed: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused 34m Warning ProbeError pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Startup probe error: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused... 34m Normal AddedLabel gateway/e2e-inf-a-891d80 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-891d80 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-33b4d7-77445cb55d Error creating: pods "payload-processing-e2e-inf-a-33b4d7-77445cb55d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-33b4d7: serviceaccount "payload-processing-e2e-inf-a-33b4d7" not found 34m Normal ScalingReplicaSet deployment/e2e-inf-a-891d80-openshift-default Scaled up replica set e2e-inf-a-891d80-openshift-default-7b9544846c from 0 to 1 34m Warning Unhealthy pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 34m Normal SuccessfulCreate replicaset/e2e-inf-a-891d80-openshift-default-7b9544846c Created pod: e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-33b4d7 Scaled up replica set payload-processing-e2e-inf-a-33b4d7-77445cb55d from 0 to 1 34m Warning ProbeError pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-33b4d7 Scaled up replica set payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-33b4d7-77445cb55d Created pod: payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw 34m Normal AddedInterface pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Add eth0 [10.132.0.134/23] from ovn-kubernetes 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c Error creating: pods "payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-33b4d7: serviceaccount "payload-processing-e2e-inf-a-33b4d7" not found 34m Normal AddedInterface pod/payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw Add eth0 [10.134.0.127/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c Created pod: payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp 34m Normal Pulled pod/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Pulled pod/payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw Container created 34m Normal Started pod/payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw Container started 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp Add eth0 [10.134.0.128/23] from ovn-kubernetes 34m Normal Created pod/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp Container created 34m Normal Started pod/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp Container started 34m Normal Killing pod/payload-processing-e2e-mt-48b1a6a7-5547c65796-crcbs Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-mt-48b1a6a7-65cdd77d95-rmdbz Stopping container payload-pre-processing 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-48b1a6a7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-48b1a6a7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-891d80 Scaled up replica set payload-processing-e2e-inf-a-891d80-78cbdfb8c6 from 0 to 1 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-8ceda0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-8ceda0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 Add eth0 [10.134.0.129/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-891d80-78cbdfb8c6 Created pod: payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-891d80 Scaled up replica set payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc Created pod: payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-891d80-78cbdfb8c6 Error creating: pods "payload-processing-e2e-inf-a-891d80-78cbdfb8c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-891d80: serviceaccount "payload-processing-e2e-inf-a-891d80" not found 34m Normal Pulled pod/payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc Error creating: pods "payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-891d80: serviceaccount "payload-processing-e2e-inf-a-891d80" not found 34m Normal AddedInterface pod/payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b Add eth0 [10.134.0.130/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b Container started 34m Normal Created pod/payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b Container created 34m Normal Created pod/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 Container created 34m Normal Started pod/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 Container started 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-9b2e27-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-9b2e27-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal SuccessfulCreate replicaset/e2e-api-b-f61e62-openshift-default-6fb69f9447 Created pod: e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt 34m Normal AddedLabel gateway/e2e-api-b-f61e62 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-f61e62 34m Normal ScalingReplicaSet deployment/e2e-api-b-f61e62-openshift-default Scaled up replica set e2e-api-b-f61e62-openshift-default-6fb69f9447 from 0 to 1 34m Normal AddedInterface pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Add eth0 [10.134.0.131/23] from ovn-kubernetes 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-f61e62 Scaled up replica set payload-processing-e2e-api-b-f61e62-5d44b7544d from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-api-b-f61e62-5d44b7544d Error creating: pods "payload-processing-e2e-api-b-f61e62-5d44b7544d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-f61e62: serviceaccount "payload-processing-e2e-api-b-f61e62" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-f61e62-5d44b7544d Created pod: payload-processing-e2e-api-b-f61e62-5d44b7544d-zkr4f 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-f61e62 Scaled up replica set payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9 Created pod: payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-mjqtx 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9 Error creating: pods "payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-f61e62: serviceaccount "payload-processing-e2e-api-b-f61e62" not found 34m Normal AddedInterface pod/payload-processing-e2e-api-b-f61e62-5d44b7544d-zkr4f Add eth0 [10.132.0.139/23] from ovn-kubernetes 34m Normal Created pod/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-mjqtx Container created 34m Normal Created pod/payload-processing-e2e-api-b-f61e62-5d44b7544d-zkr4f Container created 34m Normal Pulled pod/payload-processing-e2e-api-b-f61e62-5d44b7544d-zkr4f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Pulled pod/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-mjqtx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-mjqtx Container started 34m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-mjqtx Add eth0 [10.132.0.138/23] from ovn-kubernetes 34m Normal Started pod/payload-processing-e2e-api-b-f61e62-5d44b7544d-zkr4f Container started 34m Normal Killing pod/e2e-api-a-9a718a-openshift-default-698df85f8c-vf45k Stopping container istio-proxy 34m Normal AddedLabel gateway/e2e-api-b-8e6159 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-8e6159 34m Normal ScalingReplicaSet deployment/e2e-api-b-8e6159-openshift-default Scaled up replica set e2e-api-b-8e6159-openshift-default-dd97fcb75 from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-api-b-8e6159-openshift-default-dd97fcb75 Created pod: e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c 34m Normal AddedInterface pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Add eth0 [10.132.0.140/23] from ovn-kubernetes 34m Normal Started pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Container started 34m Normal Created pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Container created 34m Normal Pulled pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-8e6159-647c947c95 Created pod: payload-pre-processing-e2e-api-b-8e6159-647c947c95-tdxmr 34m Warning FailedCreate replicaset/payload-processing-e2e-api-b-8e6159-5fd5bcf88d Error creating: pods "payload-processing-e2e-api-b-8e6159-5fd5bcf88d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-8e6159: serviceaccount "payload-processing-e2e-api-b-8e6159" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-8e6159-5fd5bcf88d Created pod: payload-processing-e2e-api-b-8e6159-5fd5bcf88d-mmxjf 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-8e6159 Scaled up replica set payload-processing-e2e-api-b-8e6159-5fd5bcf88d from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-8e6159-647c947c95 Error creating: pods "payload-pre-processing-e2e-api-b-8e6159-647c947c95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-8e6159: serviceaccount "payload-processing-e2e-api-b-8e6159" not found 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-8e6159 Scaled up replica set payload-pre-processing-e2e-api-b-8e6159-647c947c95 from 0 to 1 34m Warning Unhealthy pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused 34m Warning ProbeError pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused... 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-48b1a6a7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Created pod/payload-processing-e2e-api-b-8e6159-5fd5bcf88d-mmxjf Container created 34m Normal Pulled pod/payload-processing-e2e-api-b-8e6159-5fd5bcf88d-mmxjf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-pre-processing-e2e-api-b-8e6159-647c947c95-tdxmr Container started 34m Normal Created pod/payload-pre-processing-e2e-api-b-8e6159-647c947c95-tdxmr Container created 34m Normal AddedInterface pod/payload-processing-e2e-api-b-8e6159-5fd5bcf88d-mmxjf Add eth0 [10.134.0.134/23] from ovn-kubernetes 34m Normal Started pod/payload-processing-e2e-api-b-8e6159-5fd5bcf88d-mmxjf Container started 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-48b1a6a7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Pulled pod/payload-pre-processing-e2e-api-b-8e6159-647c947c95-tdxmr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-8e6159-647c947c95-tdxmr Add eth0 [10.134.0.133/23] from ovn-kubernetes 33m Normal Killing pod/payload-processing-e2e-api-b-9b2e27-56c4c6f766-fphsc Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-api-b-9b2e27-5b998f9f9-8f9j5 Stopping container payload-pre-processing 33m Normal AddedLabel gateway/e2e-inf-b-afdf16 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-afdf16 33m Normal SuccessfulCreate replicaset/e2e-inf-b-afdf16-openshift-default-5c66498b5c Created pod: e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p 33m Normal ScalingReplicaSet deployment/e2e-inf-b-afdf16-openshift-default Scaled up replica set e2e-inf-b-afdf16-openshift-default-5c66498b5c from 0 to 1 33m Normal AddedInterface pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Add eth0 [10.132.0.141/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f Created pod: payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f Error creating: pods "payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-afdf16: serviceaccount "payload-processing-e2e-inf-b-afdf16" not found 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-afdf16 Scaled up replica set payload-pre-processing-e2e-inf-b-afdf16-86c778d99f from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-afdf16 Scaled up replica set payload-processing-e2e-inf-b-afdf16-7db74769f4 from 0 to 1 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-afdf16-7db74769f4 Error creating: pods "payload-processing-e2e-inf-b-afdf16-7db74769f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-afdf16: serviceaccount "payload-processing-e2e-inf-b-afdf16" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-afdf16-7db74769f4 Created pod: payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 33m Normal Pulled pod/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 Container created 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg Add eth0 [10.134.0.136/23] from ovn-kubernetes 33m Normal Pulled pod/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg Container created 33m Normal Started pod/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg Container started 33m Normal Started pod/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 Container started 33m Normal AddedInterface pod/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 Add eth0 [10.133.0.124/23] from ovn-kubernetes 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-8ceda0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-8ceda0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal ScalingReplicaSet deployment/e2e-inf-b-ef5dca-openshift-default Scaled up replica set e2e-inf-b-ef5dca-openshift-default-b7b5597b9 from 0 to 1 33m Normal AddedLabel gateway/e2e-inf-b-ef5dca Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ef5dca 33m Normal SuccessfulCreate replicaset/e2e-inf-b-ef5dca-openshift-default-b7b5597b9 Created pod: e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 33m Warning Unhealthy pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Readiness probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Readiness probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 33m Normal Pulled pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Container started 33m Normal Created pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Container created 33m Normal Created pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Container created 33m Normal Pulled pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Container started 33m Warning Unhealthy pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-33b4d7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-33b4d7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-afdf16-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-afdf16-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-891d80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-891d80-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 9; reason: All metrics below target 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-27b5d5-openshift-default New size: 4; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled down replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 10 to 9 33m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/e2e-worker-w4-27b5d5-openshift-default-66b797556c Deleted pod: e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 33m Normal Killing pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/e2e-worker-w4-27b5d5-openshift-default Scaled down replica set e2e-worker-w4-27b5d5-openshift-default-66b797556c from 5 to 4 33m Normal SuccessfulDelete replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Deleted pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-9tntv 33m Normal Killing pod/payload-processing-e2e-api-a-8ceda0-6ff4cc8588-xbn7c Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-api-a-8ceda0-54c896b567-45659 Stopping container payload-pre-processing 33m Normal Killing pod/e2e-api-b-9b2e27-openshift-default-8489bb687c-bcfjf Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-48b1a6a7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-mt-48b1a6a7-openshift-default-78f6598f5c Created pod: e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 33m Normal ScalingReplicaSet deployment/e2e-mt-48b1a6a7-openshift-default Scaled up replica set e2e-mt-48b1a6a7-openshift-default-78f6598f5c from 1 to 2 33m Normal AddedLabel gateway/e2e-inf-a-02feaa Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-02feaa 33m Normal ScalingReplicaSet deployment/e2e-inf-a-02feaa-openshift-default Scaled up replica set e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7 from 0 to 1 33m Normal SuccessfulCreate replicaset/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7 Created pod: e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj 33m Normal Pulled pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 Add eth0 [10.133.0.126/23] from ovn-kubernetes 33m Normal AddedInterface pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Add eth0 [10.132.0.142/23] from ovn-kubernetes 33m Normal Created pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 Container created 33m Normal Started pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 Container started 33m Normal Started pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Container started 33m Normal Pulled pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Container created 33m Normal Started pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Container started 33m Normal Pulled pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Container created 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-33b4d7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-33b4d7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal AddedInterface pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Add eth0 [10.134.0.138/23] from ovn-kubernetes 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5 Error creating: pods "payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ef5dca: serviceaccount "payload-processing-e2e-inf-b-ef5dca" not found 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-ef5dca Scaled up replica set payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5 from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5 Created pod: payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-ef5dca Scaled up replica set payload-processing-e2e-inf-b-ef5dca-659548cc46 from 0 to 1 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-ef5dca-659548cc46 Error creating: pods "payload-processing-e2e-inf-b-ef5dca-659548cc46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ef5dca: serviceaccount "payload-processing-e2e-inf-b-ef5dca" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-ef5dca-659548cc46 Created pod: payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 33m Normal Killing pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24 Stopping container istio-proxy 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-8e6159-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-8e6159-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-891d80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-891d80-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 7; reason: All metrics below target 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-f61e62-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-br66r Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused... 33m Normal SuccessfulDelete replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Deleted pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-f61e62-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default Scaled down replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 9 to 8 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 8; reason: All metrics below target 33m Warning ProbeError pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 33m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Deleted pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-28wrb 33m Normal SuccessfulDelete replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Deleted pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs 33m Normal SuccessfulDelete replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Deleted pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr 33m Normal SuccessfulDelete replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Deleted pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n 33m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-88jvr Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-98fjs Stopping container istio-proxy 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-02feaa-5db7cd8cd Error creating: pods "payload-processing-e2e-inf-a-02feaa-5db7cd8cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-02feaa: serviceaccount "payload-processing-e2e-inf-a-02feaa" not found 33m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled down replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 10 to 8 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-02feaa Scaled up replica set payload-processing-e2e-inf-a-02feaa-5db7cd8cd from 0 to 1 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 8; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled down replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 10 to 7 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-02feaa Scaled up replica set payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5 from 0 to 1 33m Normal SuccessfulDelete replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Deleted pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk 33m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n Stopping container istio-proxy 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5 Created pod: payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5 Error creating: pods "payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-02feaa: serviceaccount "payload-processing-e2e-inf-a-02feaa" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-02feaa-5db7cd8cd Created pod: payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh 33m Normal Created pod/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 Container created 33m Normal AddedInterface pod/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 Add eth0 [10.134.0.147/23] from ovn-kubernetes 33m Normal Pulled pod/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Pulled pod/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Started pod/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 Container started 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j Add eth0 [10.134.0.146/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j Container started 33m Normal Created pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j Container created 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q Add eth0 [10.133.0.136/23] from ovn-kubernetes 33m Normal Pulled pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q Container created 32m Normal Started pod/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q Container started 32m Normal Started pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh Container started 32m Normal Created pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh Container created 32m Normal Pulled pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh Add eth0 [10.132.0.147/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/e2e-inf-a-d52364-openshift-default Scaled up replica set e2e-inf-a-d52364-openshift-default-fcdf865d7 from 0 to 1 32m Normal AddedLabel gateway/e2e-inf-a-d52364 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-d52364 32m Normal SuccessfulCreate replicaset/e2e-inf-a-d52364-openshift-default-fcdf865d7 Created pod: e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 32m Normal AddedInterface pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Add eth0 [10.132.0.149/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-d52364-8544558f6c Created pod: payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-d52364 Scaled up replica set payload-processing-e2e-inf-a-d52364-8f7bc4cc6 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-d52364 Scaled up replica set payload-pre-processing-e2e-inf-a-d52364-8544558f6c from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-d52364-8f7bc4cc6 Created pod: payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq 32m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-d52364-8f7bc4cc6 Error creating: pods "payload-processing-e2e-inf-a-d52364-8f7bc4cc6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d52364: serviceaccount "payload-processing-e2e-inf-a-d52364" not found 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-d52364-8544558f6c Error creating: pods "payload-pre-processing-e2e-inf-a-d52364-8544558f6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d52364: serviceaccount "payload-processing-e2e-inf-a-d52364" not found 32m Normal Started pod/payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq Container started 32m Normal Created pod/payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq Container created 32m Normal Pulled pod/payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq Add eth0 [10.132.0.151/23] from ovn-kubernetes 32m Normal Pulled pod/payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh Container created 32m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh Add eth0 [10.134.0.148/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh Container started 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-02feaa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-02feaa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-8e6159-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-8e6159-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default New size: 7; reason: All metrics below target 32m Warning ProbeError pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Readiness probe error: Get "http://10.132.0.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 32m Warning Unhealthy pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Readiness probe failed: Get "http://10.132.0.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Normal SuccessfulDelete replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Deleted pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw 32m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vn5jw Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default Scaled down replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 8 to 7 32m Normal SuccessfulDelete replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Deleted pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 32m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled down replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 10 to 7 32m Normal SuccessfulDelete replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Deleted pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-26psz 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 7; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Deleted pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 32m Normal SuccessfulDelete replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Deleted pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 7; reason: All metrics below target 32m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-mhkw4 Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-sqxlj Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-rfjv7 Stopping container istio-proxy 32m Normal Created pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Container created 32m Normal Started pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Container started 32m Normal Pulled pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning ProbeError pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Startup probe error: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Startup probe failed: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused 32m Normal Killing pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Deleted pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 32m Normal SuccessfulDelete replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Deleted pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default New size: 7; reason: All metrics below target 32m Normal ScalingReplicaSet deployment/e2e-worker-w6-bce71f-openshift-default Scaled down replica set e2e-worker-w6-bce71f-openshift-default-5d99f47777 from 10 to 7 32m Normal SuccessfulDelete replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Deleted pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default New size: 4; reason: All metrics below target 32m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-slpf5 Stopping container istio-proxy 32m Normal Killing pod/payload-pre-processing-e2e-api-a-f4012a-b58495cbc-zdz5w Stopping container payload-pre-processing 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-f61e62-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-8qnr2 Stopping container istio-proxy 32m Normal Killing pod/payload-processing-e2e-api-a-f4012a-78c8dd4c94-sgr5m Stopping container payload-processing 32m Normal SuccessfulDelete replicaset/e2e-worker-w3-17e904-openshift-default-7cfdf6f655 Deleted pod: e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj 32m Normal Killing pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-qvtfj Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-xxhlw Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-worker-w3-17e904-openshift-default Scaled down replica set e2e-worker-w3-17e904-openshift-default-7cfdf6f655 from 5 to 4 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-f61e62-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning Unhealthy pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Readiness probe failed: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Normal ScalingReplicaSet deployment/e2e-inf-b-69f3a9-openshift-default Scaled up replica set e2e-inf-b-69f3a9-openshift-default-64fdcd667c from 0 to 1 32m Warning ProbeError pod/e2e-api-a-8ceda0-openshift-default-6849d5bcf-gcffr Readiness probe error: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 32m Normal AddedLabel gateway/e2e-inf-b-69f3a9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-69f3a9 32m Normal SuccessfulCreate replicaset/e2e-inf-b-69f3a9-openshift-default-64fdcd667c Created pod: e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln 32m Normal AddedInterface pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln Add eth0 [10.134.0.152/23] from ovn-kubernetes 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-33b4d7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-inf-a-33b4d7-openshift-default-5c9994896f Created pod: e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd 32m Normal ScalingReplicaSet deployment/e2e-inf-a-33b4d7-openshift-default Scaled up replica set e2e-inf-a-33b4d7-openshift-default-5c9994896f from 1 to 2 32m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-api-b-f61e62 Failed to update endpoint openshift-ingress/payload-processing-e2e-api-b-f61e62: Operation cannot be fulfilled on endpoints "payload-processing-e2e-api-b-f61e62": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-api-b-f61e62, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: c6236bfa-f3e0-4b5a-abab-0ee2a125a7f4, UID in object meta: 32m Normal Killing pod/payload-processing-e2e-api-b-f61e62-5d44b7544d-zkr4f Stopping container payload-processing 32m Normal Started pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd Container started 32m Normal AddedInterface pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd Add eth0 [10.134.0.153/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd Container created 32m Normal Killing pod/payload-pre-processing-e2e-api-b-f61e62-bdcf9d7f9-mjqtx Stopping container payload-pre-processing 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-69f3a9-765dc684fd Created pod: payload-processing-e2e-inf-b-69f3a9-765dc684fd-8rp54 32m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-69f3a9-765dc684fd Error creating: pods "payload-processing-e2e-inf-b-69f3a9-765dc684fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-69f3a9: serviceaccount "payload-processing-e2e-inf-b-69f3a9" not found 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-69f3a9 Scaled up replica set payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc Error creating: pods "payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-69f3a9: serviceaccount "payload-processing-e2e-inf-b-69f3a9" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc Created pod: payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-69f3a9 Scaled up replica set payload-processing-e2e-inf-b-69f3a9-765dc684fd from 0 to 1 32m Warning Unhealthy pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Readiness probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 32m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn Add eth0 [10.134.0.154/23] from ovn-kubernetes 32m Normal Created pod/payload-processing-e2e-inf-b-69f3a9-765dc684fd-8rp54 Container created 32m Normal Pulled pod/payload-processing-e2e-inf-b-69f3a9-765dc684fd-8rp54 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn Container started 32m Normal Pulled pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-processing-e2e-inf-b-69f3a9-765dc684fd-8rp54 Container started 32m Normal Created pod/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn Container created 32m Warning ProbeError pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Readiness probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 32m Normal AddedInterface pod/payload-processing-e2e-inf-b-69f3a9-765dc684fd-8rp54 Add eth0 [10.134.0.155/23] from ovn-kubernetes 32m Normal Pulled pod/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Container created 32m Normal Started pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Container started 32m Normal ScalingReplicaSet deployment/e2e-inf-b-c86696-openshift-default Scaled up replica set e2e-inf-b-c86696-openshift-default-859496475c from 0 to 1 32m Normal AddedLabel gateway/e2e-inf-b-c86696 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-c86696 32m Normal SuccessfulCreate replicaset/e2e-inf-b-c86696-openshift-default-859496475c Created pod: e2e-inf-b-c86696-openshift-default-859496475c-srmqr 32m Normal AddedInterface pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Add eth0 [10.134.0.156/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 31m Normal ScalingReplicaSet deployment/e2e-inf-b-ef5dca-openshift-default Scaled up replica set e2e-inf-b-ef5dca-openshift-default-b7b5597b9 from 1 to 2 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-ef5dca-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-inf-b-ef5dca-openshift-default-b7b5597b9 Created pod: e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s 31m Warning Unhealthy pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 31m Normal Pulled pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Container started 31m Normal Created pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Container created 31m Warning ProbeError pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 31m Normal AddedInterface pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Add eth0 [10.134.0.157/23] from ovn-kubernetes 31m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-c86696-88fc5bdbb Error creating: pods "payload-processing-e2e-inf-b-c86696-88fc5bdbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c86696: serviceaccount "payload-processing-e2e-inf-b-c86696" not found 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9 Error creating: pods "payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c86696: serviceaccount "payload-processing-e2e-inf-b-c86696" not found 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9 Created pod: payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-c86696 Scaled up replica set payload-pre-processing-e2e-inf-b-c86696-67f7dffff9 from 0 to 1 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-c86696 Scaled up replica set payload-processing-e2e-inf-b-c86696-88fc5bdbb from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-c86696-88fc5bdbb Created pod: payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-69f3a9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-69f3a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal Created pod/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 Container created 31m Normal AddedInterface pod/payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 Add eth0 [10.134.0.159/23] from ovn-kubernetes 31m Normal Pulled pod/payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 Container created 31m Normal Started pod/payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 Container started 31m Normal Started pod/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 Container started 31m Normal Pulled pod/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 Add eth0 [10.134.0.158/23] from ovn-kubernetes 31m Warning Unhealthy pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Readiness probe failed: Get "http://10.132.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 31m Warning ProbeError pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Readiness probe error: Get "http://10.132.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-f4012a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-f4012a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-afdf16-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-afdf16-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning Unhealthy pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Readiness probe failed: Get "http://10.132.0.149:15021/healthz/ready": read tcp 10.132.0.2:52662->10.132.0.149:15021: read: connection reset by peer 31m Normal Created pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Container created 31m Normal Pulled pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning ProbeError pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Readiness probe error: Get "http://10.132.0.149:15021/healthz/ready": read tcp 10.132.0.2:52662->10.132.0.149:15021: read: connection reset by peer... 31m Normal Started pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Container started 31m Normal Created pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Container created 31m Normal Pulled pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Container started 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ef5dca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ef5dca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning Unhealthy pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Startup probe error: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Startup probe failed: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-afdf16-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-f4012a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-f4012a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal SuccessfulCreate replicaset/e2e-inf-b-afdf16-openshift-default-5c66498b5c Created pod: e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 31m Normal ScalingReplicaSet deployment/e2e-inf-b-afdf16-openshift-default Scaled up replica set e2e-inf-b-afdf16-openshift-default-5c66498b5c from 1 to 2 31m Normal Pulled pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 Add eth0 [10.134.0.160/23] from ovn-kubernetes 31m Normal Killing pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Stopping container istio-proxy 31m Normal Created pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 Container created 31m Normal Started pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 Container started 31m Normal ScalingReplicaSet deployment/e2e-inf-a-c24c01-openshift-default Scaled up replica set e2e-inf-a-c24c01-openshift-default-7b7464ff5c from 0 to 1 31m Normal AddedLabel gateway/e2e-inf-a-c24c01 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c24c01 31m Normal SuccessfulCreate replicaset/e2e-inf-a-c24c01-openshift-default-7b7464ff5c Created pod: e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 31m Normal AddedInterface pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Add eth0 [10.134.0.161/23] from ovn-kubernetes 31m Warning ProbeError pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-17e904-openshift-default-7cfdf6f655 Deleted pod: e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj 31m Normal Killing pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-pd4lj Stopping container istio-proxy 31m Normal ScalingReplicaSet deployment/e2e-worker-w3-17e904-openshift-default Scaled down replica set e2e-worker-w3-17e904-openshift-default-7cfdf6f655 from 4 to 3 31m Warning Unhealthy pod/e2e-api-b-f61e62-openshift-default-6fb69f9447-p8jgt Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default New size: 3; reason: All metrics below target 31m Normal Killing pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 Startup probe failed: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 Startup probe error: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused... 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c24c01 Scaled up replica set payload-pre-processing-e2e-inf-a-c24c01-6679b9f99 from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c24c01-6b6df58646 Created pod: payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99 Error creating: pods "payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c24c01: serviceaccount "payload-processing-e2e-inf-a-c24c01" not found 31m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c24c01-6b6df58646 Error creating: pods "payload-processing-e2e-inf-a-c24c01-6b6df58646-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c24c01: serviceaccount "payload-processing-e2e-inf-a-c24c01" not found 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c24c01 Scaled up replica set payload-processing-e2e-inf-a-c24c01-6b6df58646 from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99 Created pod: payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v 31m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr Add eth0 [10.134.0.163/23] from ovn-kubernetes 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ef5dca-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal Started pod/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v Container started 31m Normal Pulled pod/payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v Add eth0 [10.134.0.164/23] from ovn-kubernetes 31m Normal Created pod/payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr Container created 31m Normal Created pod/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v Container created 31m Normal Started pod/payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr Container started 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ef5dca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning ProbeError pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-api-a-f4012a-openshift-default-7c884cc469-vrvmw Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d52364-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d52364-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal Killing pod/payload-pre-processing-e2e-api-b-8e6159-647c947c95-tdxmr Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-api-b-8e6159-5fd5bcf88d-mmxjf Stopping container payload-processing 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-891d80-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-inf-a-891d80-openshift-default-7b9544846c Created pod: e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 31m Normal ScalingReplicaSet deployment/e2e-inf-a-891d80-openshift-default Scaled up replica set e2e-inf-a-891d80-openshift-default-7b9544846c from 1 to 2 31m Normal AddedInterface pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Add eth0 [10.134.0.165/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Container created 31m Normal Started pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Container started 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-ef5dca-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-inf-b-ef5dca-openshift-default-b7b5597b9 Created pod: e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw 31m Normal ScalingReplicaSet deployment/e2e-inf-b-ef5dca-openshift-default Scaled up replica set e2e-inf-b-ef5dca-openshift-default-b7b5597b9 from 2 to 3 31m Normal AddedInterface pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Add eth0 [10.133.0.145/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning ProbeError pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Startup probe error: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Startup probe failed: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused 31m Normal Created pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Container created 31m Normal Started pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Container started 31m Warning Unhealthy pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Startup probe failed: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Startup probe error: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused... 30m Normal AddedLabel gateway/e2e-inf-b-76117c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-76117c 30m Normal ScalingReplicaSet deployment/e2e-inf-b-76117c-openshift-default Scaled up replica set e2e-inf-b-76117c-openshift-default-5bf558ddbc from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-inf-b-76117c-openshift-default-5bf558ddbc Created pod: e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-02feaa-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 30m Normal Pulled pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Container created 30m Normal AddedInterface pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w Add eth0 [10.134.0.168/23] from ovn-kubernetes 30m Normal Started pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Container started 30m Normal Started pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln Container started 30m Normal Created pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln Container created 30m Normal Pulled pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal ScalingReplicaSet deployment/e2e-inf-a-02feaa-openshift-default Scaled up replica set e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7 from 1 to 3 30m Normal SuccessfulCreate replicaset/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7 Created pod: e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz 30m Normal SuccessfulCreate replicaset/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7 Created pod: e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 30m Normal Started pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Container started 30m Normal AddedInterface pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Add eth0 [10.134.0.169/23] from ovn-kubernetes 30m Normal Created pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Container created 30m Normal Pulled pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz Add eth0 [10.132.0.157/23] from ovn-kubernetes 30m Normal Started pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz Container started 30m Normal Pulled pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz Container created 30m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-76117c-7b6dc4bd8 Error creating: pods "payload-processing-e2e-inf-b-76117c-7b6dc4bd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-76117c: serviceaccount "payload-processing-e2e-inf-b-76117c" not found 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859 Error creating: pods "payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-76117c: serviceaccount "payload-processing-e2e-inf-b-76117c" not found 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-76117c Scaled up replica set payload-processing-e2e-inf-b-76117c-7b6dc4bd8 from 0 to 1 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-76117c Scaled up replica set payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859 from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-76117c-7b6dc4bd8 Created pod: payload-processing-e2e-inf-b-76117c-7b6dc4bd8-tjgr5 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859 Created pod: payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd 30m Warning Unhealthy pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz Startup probe error: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz Startup probe failed: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused... 30m Normal Pulled pod/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Warning ProbeError pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused... 30m Normal Created pod/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd Container created 30m Normal Started pod/payload-processing-e2e-inf-b-76117c-7b6dc4bd8-tjgr5 Container started 30m Normal Created pod/payload-processing-e2e-inf-b-76117c-7b6dc4bd8-tjgr5 Container created 30m Normal Started pod/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd Container started 30m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd Add eth0 [10.134.0.170/23] from ovn-kubernetes 30m Normal Pulled pod/payload-processing-e2e-inf-b-76117c-7b6dc4bd8-tjgr5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-inf-b-76117c-7b6dc4bd8-tjgr5 Add eth0 [10.134.0.171/23] from ovn-kubernetes 30m Warning Unhealthy pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused 30m Normal Killing pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Stopping container istio-proxy 30m Warning ProbeError pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Startup probe error: Get "http://10.134.0.169:15021/healthz/ready": dial tcp 10.134.0.169:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Startup probe failed: Get "http://10.134.0.169:15021/healthz/ready": dial tcp 10.134.0.169:15021: connect: connection refused 30m Normal AddedLabel gateway/e2e-inf-a-f2dbd9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-f2dbd9 30m Normal ScalingReplicaSet deployment/e2e-inf-a-f2dbd9-openshift-default Scaled up replica set e2e-inf-a-f2dbd9-openshift-default-776ff48945 from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-inf-a-f2dbd9-openshift-default-776ff48945 Created pod: e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d 30m Normal AddedInterface pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Add eth0 [10.134.0.172/23] from ovn-kubernetes 30m Warning ProbeError pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-api-b-8e6159-openshift-default-dd97fcb75-hvm7c Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-76117c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-f2dbd9 Scaled up replica set payload-processing-e2e-inf-a-f2dbd9-7b9996c75d from 0 to 1 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-02feaa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-02feaa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-76117c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-f2dbd9 Scaled up replica set payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f from 0 to 1 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f Error creating: pods "payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f2dbd9: serviceaccount "payload-processing-e2e-inf-a-f2dbd9" not found 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f Created pod: payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz 30m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d Error creating: pods "payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f2dbd9: serviceaccount "payload-processing-e2e-inf-a-f2dbd9" not found 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d Created pod: payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb 30m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz Add eth0 [10.132.0.158/23] from ovn-kubernetes 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-69f3a9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal Pulled pod/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Created pod/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz Container created 30m Normal Started pod/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz Container started 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-69f3a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal AddedInterface pod/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb Add eth0 [10.132.0.159/23] from ovn-kubernetes 30m Normal Pulled pod/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Created pod/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb Container created 30m Normal Killing pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 Stopping container istio-proxy 30m Normal Started pod/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb Container started 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-b281d5-openshift-default Scaled down replica set e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5 from 4 to 3 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-b281d5-openshift-default New size: 3; reason: All metrics below target 30m Normal SuccessfulDelete replicaset/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5 Deleted pod: e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-7g548 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-891d80-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-inf-a-891d80-openshift-default Scaled up replica set e2e-inf-a-891d80-openshift-default-7b9544846c from 2 to 3 30m Normal SuccessfulCreate replicaset/e2e-inf-a-891d80-openshift-default-7b9544846c Created pod: e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 30m Normal Created pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Container created 30m Normal Started pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Container started 30m Normal Pulled pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Add eth0 [10.132.0.160/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Container created 30m Normal Started pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Container started 30m Warning ProbeError pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Startup probe error: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Startup probe failed: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-69f3a9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal SuccessfulCreate replicaset/e2e-inf-b-69f3a9-openshift-default-64fdcd667c Created pod: e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d 30m Normal ScalingReplicaSet deployment/e2e-inf-b-69f3a9-openshift-default Scaled up replica set e2e-inf-b-69f3a9-openshift-default-64fdcd667c from 1 to 2 30m Normal Started pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d Container started 30m Normal Pulled pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d Container created 30m Normal AddedInterface pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d Add eth0 [10.132.0.162/23] from ovn-kubernetes 30m Warning ProbeError pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Startup probe error: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Startup probe failed: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused 30m Normal SuccessfulCreate replicaset/e2e-inf-a-d52364-openshift-default-fcdf865d7 Created pod: e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf 30m Normal ScalingReplicaSet deployment/e2e-inf-a-d52364-openshift-default Scaled up replica set e2e-inf-a-d52364-openshift-default-fcdf865d7 from 1 to 2 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-d52364-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal Started pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf Container started 30m Normal AddedInterface pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf Add eth0 [10.132.0.163/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf Container created 30m Warning Unhealthy pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c86696-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c86696-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning Unhealthy pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused... 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c86696-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c86696-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal Started pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w Container started 29m Normal Created pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w Container created 29m Normal Pulled pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-02feaa-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulCreate replicaset/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7 Created pod: e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c 29m Normal ScalingReplicaSet deployment/e2e-inf-a-02feaa-openshift-default Scaled up replica set e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7 from 3 to 4 29m Warning ProbeError pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w Startup probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w Startup probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e3586f-openshift-default Scaled up replica set e2e-shared-b-w1-e3586f-openshift-default-976787cfc from 1 to 2 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-e3586f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e3586f-openshift-default-976787cfc Created pod: e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 29m Normal Killing pod/payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 Stopping container payload-pre-processing 29m Normal Pulled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 Add eth0 [10.132.0.167/23] from ovn-kubernetes 29m Normal Created pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 Container created 29m Normal Started pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 Container started 29m Warning Unhealthy pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Readiness probe failed: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning ProbeError pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Readiness probe error: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-76117c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Pulled pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Killing pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Stopping container istio-proxy 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-76117c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Killing pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 Startup probe failed: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused 28m Normal Killing pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Stopping container istio-proxy 28m Warning ProbeError pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 Startup probe error: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused... 28m Normal Created pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Container created 28m Normal Started pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Container started 28m Warning Unhealthy pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal SuccessfulCreate replicaset/e2e-inf-b-69f3a9-openshift-default-64fdcd667c Created pod: e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-69f3a9-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-inf-b-69f3a9-openshift-default Scaled up replica set e2e-inf-b-69f3a9-openshift-default-64fdcd667c from 2 to 3 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-891d80-5959cbd4dc-4snm2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning ProbeError pod/e2e-inf-a-891d80-openshift-default-7b9544846c-76fz6 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/payload-processing-e2e-inf-a-891d80-78cbdfb8c6-9488b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw Stopping container payload-processing 28m Warning Unhealthy pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-a-891d80-openshift-default-7b9544846c-pxkg2 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-a-891d80-openshift-default-7b9544846c-z4592 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal Created pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c Container created 28m Normal AddedInterface pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c Add eth0 [10.132.0.168/23] from ovn-kubernetes 28m Normal AddedInterface pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk Add eth0 [10.134.0.179/23] from ovn-kubernetes 28m Normal Created pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk Container created 28m Normal Started pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c Container started 28m Normal Pulled pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk Container started 28m Normal Pulled pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c24c01-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c24c01-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-618d93-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q Stopping container payload-pre-processing 28m Normal SuccessfulCreate replicaset/e2e-inf-b-c86696-openshift-default-859496475c Created pod: e2e-inf-b-c86696-openshift-default-859496475c-9ttpm 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-618d93-openshift-default-79769f779 Created pod: e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-c86696-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-inf-b-c86696-openshift-default Scaled up replica set e2e-inf-b-c86696-openshift-default-859496475c from 1 to 2 28m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-inf-b-ef5dca Failed to update endpoint openshift-ingress/payload-processing-e2e-inf-b-ef5dca: Operation cannot be fulfilled on endpoints "payload-processing-e2e-inf-b-ef5dca": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-inf-b-ef5dca, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 680bb5f7-7c18-462d-897f-2dde47d7978b, UID in object meta: 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-618d93-openshift-default Scaled up replica set e2e-shared-a-w1-618d93-openshift-default-79769f779 from 1 to 2 28m Normal Killing pod/payload-processing-e2e-inf-b-ef5dca-659548cc46-g8lr4 Stopping container payload-processing 28m Normal Created pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs Container created 28m Normal Started pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs Container started 28m Normal Pulled pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs Add eth0 [10.132.0.171/23] from ovn-kubernetes 28m Warning ProbeError pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk Startup probe error: Get "http://10.134.0.179:15021/healthz/ready": dial tcp 10.134.0.179:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk Startup probe failed: Get "http://10.134.0.179:15021/healthz/ready": dial tcp 10.134.0.179:15021: connect: connection refused 28m Warning Unhealthy pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c Startup probe failed: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused 28m Normal AddedInterface pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Add eth0 [10.133.0.147/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Container created 28m Warning ProbeError pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c Startup probe error: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused... 28m Normal Started pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Container started 28m Normal Killing pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-72vxd Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-a-33b4d7-openshift-default-5c9994896f-hvxlt Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Startup probe failed: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused 28m Warning Unhealthy pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 28m Warning ProbeError pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Startup probe error: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused... 28m Normal Killing pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-gbx9s Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal Killing pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-z8tl6 Stopping container istio-proxy 28m Normal ScalingReplicaSet deployment/e2e-inf-b-1fa500-openshift-default Scaled up replica set e2e-inf-b-1fa500-openshift-default-5f684fd55f from 0 to 1 28m Normal AddedLabel gateway/e2e-inf-b-1fa500 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-1fa500 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-69c573-openshift-default Scaled up replica set e2e-shared-a-w6-69c573-openshift-default-95546bd69 from 0 to 1 28m Normal AddedLabel gateway/e2e-shared-a-w6-69c573 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-69c573 28m Normal SuccessfulCreate replicaset/e2e-inf-b-1fa500-openshift-default-5f684fd55f Created pod: e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq 28m Warning ProbeError pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v 28m Normal AddedInterface pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Add eth0 [10.134.0.183/23] from ovn-kubernetes 28m Normal AddedInterface pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Add eth0 [10.134.0.184/23] from ovn-kubernetes 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-33b4d7-5957cfbf6c-trlrp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg Stopping container payload-pre-processing 28m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-inf-b-afdf16 Failed to update endpoint openshift-ingress/payload-processing-e2e-inf-b-afdf16: Operation cannot be fulfilled on endpoints "payload-processing-e2e-inf-b-afdf16": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-inf-b-afdf16, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b06b622a-4e81-4e4f-ba1d-012dd0b7a50b, UID in object meta: 28m Normal Killing pod/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 Stopping container payload-processing 28m Warning Unhealthy pod/payload-processing-e2e-inf-a-33b4d7-77445cb55d-h2jzw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56 Error creating: pods "payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1fa500: serviceaccount "payload-processing-e2e-inf-b-1fa500" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-1fa500 Scaled up replica set payload-processing-e2e-inf-b-1fa500-6cb9b7dc56 from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659 Created pod: payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-1fa500 Scaled up replica set payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659 from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659 Error creating: pods "payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1fa500: serviceaccount "payload-processing-e2e-inf-b-1fa500" not found 28m Warning Unhealthy pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Readiness probe failed: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Warning ProbeError pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Readiness probe error: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56 Created pod: payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c Created pod: payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-69c573 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-69c573 Scaled up replica set payload-processing-e2e-shared-a-w6-69c573-689db57bf8 from 0 to 1 28m Normal Pulled pod/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp Add eth0 [10.134.0.186/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f Container started 28m Normal Created pod/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f Container created 28m Normal Pulled pod/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f Add eth0 [10.134.0.185/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-69c573-689db57bf8 Created pod: payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s 28m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-69c573-689db57bf8 Error creating: pods "payload-processing-e2e-shared-a-w6-69c573-689db57bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-69c573: serviceaccount "payload-processing-e2e-shared-a-w6-69c573" not found 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c Error creating: pods "payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-69c573: serviceaccount "payload-processing-e2e-shared-a-w6-69c573" not found 28m Normal Created pod/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp Container created 28m Normal Started pod/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp Container started 28m Normal Started pod/payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s Container started 28m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s Add eth0 [10.134.0.188/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt Add eth0 [10.134.0.187/23] from ovn-kubernetes 28m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt Container started 28m Normal Created pod/payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s Container created 28m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt Container created 28m Normal Pulled pod/payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Started pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Container started 28m Normal Created pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Container created 28m Normal Pulled pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c24c01-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c24c01-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning Unhealthy pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused 28m Warning ProbeError pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused... 28m Normal SuccessfulDelete replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Deleted pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 5; reason: All metrics below target 28m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs Stopping container istio-proxy 28m Normal ScalingReplicaSet deployment/e2e-worker-w4-27b5d5-openshift-default Scaled down replica set e2e-worker-w4-27b5d5-openshift-default-66b797556c from 4 to 3 28m Normal SuccessfulDelete replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Deleted pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp 28m Normal SuccessfulDelete replicaset/e2e-worker-w4-27b5d5-openshift-default-66b797556c Deleted pod: e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b 28m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 Stopping container istio-proxy 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 6; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled down replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 7 to 6 28m Normal SuccessfulDelete replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Deleted pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g 28m Normal Killing pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-bgt7b Stopping container istio-proxy 28m Normal SuccessfulDelete replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Deleted pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-g8hrs 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-27b5d5-openshift-default New size: 3; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled down replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 7 to 5 28m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp Stopping container istio-proxy 28m Normal SuccessfulDelete replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Deleted pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default New size: 6; reason: All metrics below target 28m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-c7jqw Stopping container istio-proxy 28m Warning Unhealthy pod/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-afdf16-86c778d99f-xr7wg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal SuccessfulCreate replicaset/e2e-inf-b-76117c-openshift-default-5bf558ddbc Created pod: e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-76117c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh Stopping container payload-processing 28m Normal ScalingReplicaSet deployment/e2e-inf-b-76117c-openshift-default Scaled up replica set e2e-inf-b-76117c-openshift-default-5bf558ddbc from 1 to 2 28m Normal Pulled pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Add eth0 [10.132.0.175/23] from ovn-kubernetes 28m Normal Created pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Container created 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh Stopping container payload-pre-processing 28m Normal Started pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Container started 28m Normal Killing pod/payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq Stopping container payload-processing 28m Normal Killing pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-ltsg9 Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Stopping container istio-proxy 28m Normal AddedLabel gateway/e2e-shared-a-w3-ddb6ea Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-ddb6ea 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-ddb6ea-openshift-default-c489869 Created pod: e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-ddb6ea-openshift-default Scaled up replica set e2e-shared-a-w3-ddb6ea-openshift-default-c489869 from 0 to 1 28m Normal AddedInterface pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 Add eth0 [10.132.0.176/23] from ovn-kubernetes 28m Warning ProbeError pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-b-afdf16-openshift-default-5c66498b5c-q886p Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Startup probe failed: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused 28m Warning ProbeError pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Startup probe error: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused... 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-69c573-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979 Created pod: payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-ddb6ea: serviceaccount "payload-processing-e2e-shared-a-w3-ddb6ea" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b Created pod: payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-ddb6ea Scaled up replica set payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979 from 0 to 1 28m Normal AddedInterface pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Add eth0 [10.132.0.177/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/e2e-inf-a-c24c01-openshift-default Scaled up replica set e2e-inf-a-c24c01-openshift-default-7b7464ff5c from 1 to 2 28m Normal SuccessfulCreate replicaset/e2e-inf-a-c24c01-openshift-default-7b7464ff5c Created pod: e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-69c573-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-c24c01-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b Error creating: pods "payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-ddb6ea: serviceaccount "payload-processing-e2e-shared-a-w3-ddb6ea" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-ddb6ea Scaled up replica set payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b from 0 to 1 28m Normal Created pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Container created 28m Normal Pulled pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp Container started 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f2dbd9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f2dbd9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr Container started 28m Normal Created pod/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp Container created 28m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr Container created 28m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr Add eth0 [10.134.0.191/23] from ovn-kubernetes 28m Normal Pulled pod/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp Add eth0 [10.134.0.192/23] from ovn-kubernetes 28m Normal Started pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Container started 28m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-02feaa-77cc96dff5-nn57j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Startup probe failed: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused 27m Warning ProbeError pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Startup probe error: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused... 27m Warning Unhealthy pod/payload-processing-e2e-inf-a-d52364-8f7bc4cc6-fvcdq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-inf-a-02feaa-5db7cd8cd-vq8rh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-d52364-8544558f6c-v5fvh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-qztpf Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-a-d52364-openshift-default-fcdf865d7-pqcc9 Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Stopping container istio-proxy 27m Warning ProbeError pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-jzhwz Stopping container istio-proxy 27m Warning Unhealthy pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-xf8vj Readiness probe failed: HTTP probe failed with statuscode: 503 27m Normal Killing pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-56g9c Stopping container istio-proxy 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f2dbd9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f2dbd9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning Unhealthy pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-inf-a-02feaa-openshift-default-7b4c9dd4c7-th9n2 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 5; reason: All metrics below target 27m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled down replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 6 to 5 27m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/e2e-worker-w6-bce71f-openshift-default Scaled down replica set e2e-worker-w6-bce71f-openshift-default-5d99f47777 from 7 to 6 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default New size: 6; reason: All metrics below target 27m Normal SuccessfulDelete replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Deleted pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 27m Normal SuccessfulDelete replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Deleted pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 27m Normal SuccessfulDelete replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Deleted pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb 27m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Stopping container istio-proxy 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf Created pod: e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0a6e66-openshift-default Scaled up replica set e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf from 0 to 1 27m Normal AddedLabel gateway/e2e-shared-b-w6-0a6e66 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-0a6e66 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 Stopping container payload-processing 27m Normal AddedInterface pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Add eth0 [10.132.0.178/23] from ovn-kubernetes 27m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 27m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-4hs86 Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-0a6e66: serviceaccount "payload-processing-e2e-shared-b-w6-0a6e66" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-0a6e66 Scaled up replica set payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94 Created pod: payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 27m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94 Error creating: pods "payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-0a6e66: serviceaccount "payload-processing-e2e-shared-b-w6-0a6e66" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47 Created pod: payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-0a6e66 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47 from 0 to 1 27m Normal Created pod/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 Container created 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn Stopping container payload-pre-processing 27m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt Container created 27m Normal Started pod/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 Container started 27m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt Add eth0 [10.132.0.179/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 Add eth0 [10.132.0.180/23] from ovn-kubernetes 27m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt Container started 27m Normal Killing pod/payload-processing-e2e-inf-b-69f3a9-765dc684fd-8rp54 Stopping container payload-processing 27m Normal Pulled pod/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-0a6e66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-0a6e66-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Killing pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Stopping container istio-proxy 27m Warning ProbeError pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-inf-b-c86696-openshift-default-859496475c-srmqr Readiness probe failed: HTTP probe failed with statuscode: 503 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-8e6762-openshift-default Scaled up replica set e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 from 0 to 1 27m Normal AddedLabel gateway/e2e-shared-a-w4-8e6762 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-8e6762 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 Created pod: e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc 27m Normal AddedInterface pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Add eth0 [10.134.0.196/23] from ovn-kubernetes 27m Warning Unhealthy pod/payload-processing-e2e-inf-b-c86696-88fc5bdbb-d8nq8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Created pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Container created 27m Normal Started pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Container started 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-c86696-67f7dffff9-nm7h8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Pulled pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Warning Unhealthy pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-fd7c77-openshift-default Scaled up replica set e2e-shared-b-w3-fd7c77-openshift-default-6995885647 from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 27m Normal AddedLabel gateway/e2e-shared-b-w3-fd7c77 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-fd7c77 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8 Created pod: payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-8e6762 Scaled up replica set payload-processing-e2e-shared-a-w4-8e6762-7745b85d8 from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8 Error creating: pods "payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-8e6762: serviceaccount "payload-processing-e2e-shared-a-w4-8e6762" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-8e6762 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695 Created pod: payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-8e6762: serviceaccount "payload-processing-e2e-shared-a-w4-8e6762" not found 27m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Add eth0 [10.134.0.197/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/e2e-worker-w6-bce71f-openshift-default Scaled down replica set e2e-worker-w6-bce71f-openshift-default-5d99f47777 from 6 to 5 27m Normal SuccessfulDelete replicaset/e2e-worker-w6-bce71f-openshift-default-5d99f47777 Deleted pod: e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l 27m Normal Created pod/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 Container created 27m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 Container started 27m Normal Started pod/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 Container started 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-bce71f-openshift-default New size: 5; reason: All metrics below target 27m Normal Pulled pod/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 Add eth0 [10.133.0.152/23] from ovn-kubernetes 27m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 Container created 27m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Stopping container istio-proxy 27m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 Add eth0 [10.132.0.183/23] from ovn-kubernetes 27m Warning Unhealthy pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Startup probe failed: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused 27m Warning ProbeError pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Startup probe error: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused... 27m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lb25l Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744 Error creating: pods "payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-fd7c77: serviceaccount "payload-processing-e2e-shared-b-w3-fd7c77" not found 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744 Created pod: payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-fd7c77 Scaled up replica set payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744 from 0 to 1 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-69f3a9-5744d86bc-ljlpn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf Error creating: pods "payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-fd7c77: serviceaccount "payload-processing-e2e-shared-b-w3-fd7c77" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf Created pod: payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-fd7c77 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf from 0 to 1 27m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 Container created 27m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 Add eth0 [10.133.0.153/23] from ovn-kubernetes 27m Normal Killing pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-pk7nk Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-nqzln Stopping container istio-proxy 27m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn Add eth0 [10.134.0.198/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Killing pod/e2e-inf-b-69f3a9-openshift-default-64fdcd667c-cdb5d Stopping container istio-proxy 27m Normal Created pod/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn Container created 27m Normal Started pod/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn Container started 27m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 Container started 27m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-1fa500-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-1fa500-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-855f9574cf from 10 to 9 27m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: All metrics below target 27m Normal Killing pod/maas-default-gateway-openshift-default-855f9574cf-fpplz Stopping container istio-proxy 27m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-855f9574cf Deleted pod: maas-default-gateway-openshift-default-855f9574cf-fpplz 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-8e6762-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-8e6762-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-1fa500-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-1fa500-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-fd7c77-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-fd7c77-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal SuccessfulDelete replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Deleted pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq 26m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-d2qvq Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-76117c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal ScalingReplicaSet deployment/e2e-inf-b-76117c-openshift-default Scaled up replica set e2e-inf-b-76117c-openshift-default-5bf558ddbc from 2 to 3 26m Normal SuccessfulCreate replicaset/e2e-inf-b-76117c-openshift-default-5bf558ddbc Created pod: e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf Created pod: e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf Created pod: e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0a6e66-openshift-default Scaled up replica set e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf from 1 to 4 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-0a6e66-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf Created pod: e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 26m Normal Created pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Container created 26m Normal Pulled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 Container created 26m Normal Pulled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq Add eth0 [10.134.0.201/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 Add eth0 [10.134.0.202/23] from ovn-kubernetes 26m Normal Started pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq Container started 26m Normal AddedInterface pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w Add eth0 [10.133.0.156/23] from ovn-kubernetes 26m Normal Started pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 Container started 26m Normal Created pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w Container created 26m Normal Started pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w Container started 26m Normal Started pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Container started 26m Normal Pulled pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Add eth0 [10.133.0.157/23] from ovn-kubernetes 26m Normal Created pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq Container created 26m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Readiness probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Readiness probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused 26m Normal Created pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Container created 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-ddb6ea-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Pulled pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-ddb6ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Pulled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Container started 26m Normal Created pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 Container created 26m Normal Started pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 Container started 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-1b1531-openshift-default Scaled up replica set e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 from 0 to 1 26m Normal AddedLabel gateway/e2e-shared-b-w4-1b1531 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-1b1531 26m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Add eth0 [10.134.0.203/23] from ovn-kubernetes 26m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused... 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e3586f-openshift-default Scaled up replica set e2e-shared-b-w1-e3586f-openshift-default-976787cfc from 2 to 4 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e3586f-openshift-default-976787cfc Created pod: e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e3586f-openshift-default-976787cfc Created pod: e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 26m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused 26m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w Startup probe failed: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w Startup probe error: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused... 26m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 Startup probe error: Get "http://10.134.0.202:15021/healthz/ready": dial tcp 10.134.0.202:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 Startup probe failed: Get "http://10.134.0.202:15021/healthz/ready": dial tcp 10.134.0.202:15021: connect: connection refused 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-e3586f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 26m Warning Unhealthy pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Startup probe failed: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused 26m Normal Created pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq Container created 26m Normal Started pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq Container started 26m Normal AddedInterface pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq Add eth0 [10.133.0.158/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Warning ProbeError pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Startup probe error: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused... 26m Normal Pulled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 Add eth0 [10.132.0.185/23] from ovn-kubernetes 26m Normal Created pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 Container created 26m Normal Started pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 Container started 26m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Startup probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused... 26m Warning ProbeError pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 Startup probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 Startup probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 26m Normal ScalingReplicaSet deployment/e2e-inf-a-f2dbd9-openshift-default Scaled up replica set e2e-inf-a-f2dbd9-openshift-default-776ff48945 from 1 to 2 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-f2dbd9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 26m Normal SuccessfulCreate replicaset/e2e-inf-a-f2dbd9-openshift-default-776ff48945 Created pod: e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr 26m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Startup probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused 26m Normal ScalingReplicaSet deployment/e2e-worker-w3-17e904-openshift-default Scaled down replica set e2e-worker-w3-17e904-openshift-default-7cfdf6f655 from 3 to 2 26m Normal SuccessfulDelete replicaset/e2e-worker-w3-17e904-openshift-default-7cfdf6f655 Deleted pod: e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-1b1531 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f from 0 to 1 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-17e904-openshift-default New size: 2; reason: All metrics below target 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-1b1531 Scaled up replica set payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b from 0 to 1 26m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-z9mj5 Add eth0 [10.134.0.205/23] from ovn-kubernetes 26m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b Error creating: pods "payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-1b1531: serviceaccount "payload-processing-e2e-shared-b-w4-1b1531" not found 26m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-z9mj5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-z9mj5 Container created 26m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-z9mj5 Container started 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f Error creating: pods "payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-1b1531: serviceaccount "payload-processing-e2e-shared-b-w4-1b1531" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f Created pod: payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-z9mj5 26m Normal AddedInterface pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Add eth0 [10.132.0.186/23] from ovn-kubernetes 26m Normal Killing pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-sjn2z Stopping container istio-proxy 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b Created pod: payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 26m Normal Pulled pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 Add eth0 [10.134.0.204/23] from ovn-kubernetes 26m Normal Created pod/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 Container created 26m Normal Started pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Container started 26m Normal Started pod/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 Container started 26m Normal Created pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Container created 26m Normal Pulled pod/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Warning Unhealthy pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq Startup probe failed: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq Startup probe error: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused... 26m Warning ProbeError pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 26m Warning Unhealthy pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Startup probe failed: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused 26m Warning ProbeError pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Startup probe error: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused... 26m Normal ScalingReplicaSet deployment/e2e-inf-b-1fa500-openshift-default Scaled up replica set e2e-inf-b-1fa500-openshift-default-5f684fd55f from 1 to 2 26m Normal SuccessfulCreate replicaset/e2e-inf-b-1fa500-openshift-default-5f684fd55f Created pod: e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-1fa500-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 26m Normal Started pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Container started 26m Normal Created pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Container created 26m Normal AddedInterface pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Add eth0 [10.132.0.189/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-0a6e66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning ProbeError pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-0a6e66-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-ddb6ea-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-ddb6ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-1b1531-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-1b1531-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-fd7c77-openshift-default Scaled up replica set e2e-shared-b-w3-fd7c77-openshift-default-6995885647 from 1 to 4 25m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg Add eth0 [10.132.0.191/23] from ovn-kubernetes 25m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d Add eth0 [10.133.0.160/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d Container created 25m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg Container created 25m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg Container started 25m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p Add eth0 [10.134.0.209/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-fd7c77-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p Container started 25m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p Container created 25m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d Container started 25m Normal Killing pod/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v Stopping container payload-pre-processing 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p Startup probe failed: Get "http://10.134.0.209:15021/healthz/ready": dial tcp 10.134.0.209:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p Startup probe error: Get "http://10.134.0.209:15021/healthz/ready": dial tcp 10.134.0.209:15021: connect: connection refused... 25m Normal Killing pod/payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr Stopping container payload-processing 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d Startup probe failed: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg Startup probe error: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg Startup probe failed: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d Startup probe error: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused... 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Created pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp 25m Normal Created pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Container created 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-2405fb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal Pulled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-2405fb-openshift-default Scaled up replica set e2e-shared-a-w0-2405fb-openshift-default-556d785b45 from 1 to 2 25m Normal Started pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Container started 25m Normal AddedInterface pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Add eth0 [10.132.0.192/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 Created pod: e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 Created pod: e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 Created pod: e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-8e6762-openshift-default Scaled up replica set e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 from 1 to 4 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-8e6762-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq Add eth0 [10.133.0.161/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq Container started 25m Normal Created pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq Container created 25m Normal AddedInterface pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Add eth0 [10.134.0.210/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 Add eth0 [10.132.0.193/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 Container started 25m Normal Created pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 Container created 25m Normal Created pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Container created 25m Normal Started pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Container started 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-69c573-openshift-default Scaled up replica set e2e-shared-a-w6-69c573-openshift-default-95546bd69 from 1 to 4 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-69c573-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt 25m Normal Pulled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-fd7c77-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-fd7c77-openshift-default Scaled up replica set e2e-shared-b-w3-fd7c77-openshift-default-6995885647 from 4 to 8 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw 25m Normal AddedInterface pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd Add eth0 [10.132.0.195/23] from ovn-kubernetes 25m Normal Created pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Container created 25m Normal Pulled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm Add eth0 [10.132.0.196/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Add eth0 [10.134.0.211/23] from ovn-kubernetes 25m Normal Started pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Container started 25m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq Startup probe error: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Startup probe failed: Get "http://10.134.0.210:15021/healthz/ready": dial tcp 10.134.0.210:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq Startup probe failed: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused 25m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Add eth0 [10.132.0.197/23] from ovn-kubernetes 25m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Add eth0 [10.134.0.212/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Startup probe error: Get "http://10.134.0.210:15021/healthz/ready": dial tcp 10.134.0.210:15021: connect: connection refused... 25m Normal Created pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd Container created 25m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Startup probe error: Get "http://10.132.0.192:15021/healthz/ready": dial tcp 10.132.0.192:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Startup probe failed: Get "http://10.132.0.192:15021/healthz/ready": dial tcp 10.132.0.192:15021: connect: connection refused 25m Normal Created pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm Container created 25m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Container created 25m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Readiness probe error: Get "http://10.134.0.184:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Normal Started pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm Container started 25m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Readiness probe failed: Get "http://10.134.0.184:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Normal Started pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd Container started 25m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Readiness probe failed: Get "http://10.134.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Container started 25m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 Startup probe failed: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused 25m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn Readiness probe failed: timeout: health rpc did not complete within 1s 25m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Container created 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Readiness probe error: Get "http://10.134.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 Startup probe error: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused... 25m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Container started 25m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Startup probe failed: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Startup probe error: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused... 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 Created pod: e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-8e6762-openshift-default Scaled up replica set e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 from 4 to 8 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 Created pod: e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 Created pod: e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-8e6762-openshift-default New size: 8; reason: 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79 Created pod: e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Startup probe error: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd Startup probe failed: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd Startup probe error: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Startup probe failed: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused 25m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Add eth0 [10.134.0.214/23] from ovn-kubernetes 25m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm Startup probe failed: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused 25m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm Startup probe error: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused... 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Startup probe error: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Startup probe failed: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused 25m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Container started 25m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Container created 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-s5pxv 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-pgdtb 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-69c573-openshift-default Scaled up replica set e2e-shared-a-w6-69c573-openshift-default-95546bd69 from 4 to 8 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-69c573-openshift-default New size: 8; reason: 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-8fk7t 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-fd7c77-openshift-default Scaled up replica set e2e-shared-b-w3-fd7c77-openshift-default-6995885647 from 8 to 10 25m Normal Pulled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-fd7c77-openshift-default New size: 10; reason: 25m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Container started 25m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Container created 25m Normal Started pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Container started 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 (combined from similar events): Created pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 25m Normal Created pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Container created 25m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c24c01-6679b9f99-zf58v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-processing-e2e-inf-a-c24c01-6b6df58646-phxcr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Stopping container istio-proxy 25m Normal Killing pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-pksh8 Stopping container istio-proxy 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Startup probe failed: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Startup probe error: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused... 25m Normal Pulled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/e2e-inf-b-76117c-openshift-default Scaled up replica set e2e-inf-b-76117c-openshift-default-5bf558ddbc from 3 to 4 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-76117c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-inf-b-76117c-openshift-default-5bf558ddbc Created pod: e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 25m Normal Created pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Container created 25m Normal Started pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Container started 25m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw Add eth0 [10.134.0.216/23] from ovn-kubernetes 25m Warning ProbeError pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Readiness probe error: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Startup probe failed: Get "http://10.134.0.197:15021/healthz/ready": dial tcp 10.134.0.197:15021: connect: connection refused 25m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Startup probe error: Get "http://10.134.0.197:15021/healthz/ready": dial tcp 10.134.0.197:15021: connect: connection refused... 25m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw Container started 25m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw Container created 25m Warning Unhealthy pod/e2e-inf-a-c24c01-openshift-default-7b7464ff5c-fbc4x Readiness probe failed: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused 25m Normal Created pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Container created 25m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Startup probe error: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Startup probe failed: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused 25m Normal AddedInterface pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Add eth0 [10.132.0.199/23] from ovn-kubernetes 25m Normal Started pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Container started 25m Normal Pulled pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-8e6762-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-8e6762-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-ddb6ea-openshift-default-c489869 Created pod: e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-ddb6ea-openshift-default Scaled up replica set e2e-shared-a-w3-ddb6ea-openshift-default-c489869 from 1 to 2 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-ddb6ea-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 25m Normal AddedInterface pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Add eth0 [10.134.0.217/23] from ovn-kubernetes 25m Normal Started pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Container started 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-1b1531-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-1b1531-openshift-default Scaled up replica set e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 from 1 to 4 25m Normal Created pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Container created 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz 25m Normal Pulled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Startup probe failed: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused 25m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-inf-b-76117c Failed to update endpoint openshift-ingress/payload-processing-e2e-inf-b-76117c: Operation cannot be fulfilled on endpoints "payload-processing-e2e-inf-b-76117c": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-inf-b-76117c, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 868154f0-ce4e-4bfa-b5ba-ea4b5c3c97fa, UID in object meta: 25m Normal Killing pod/payload-processing-e2e-inf-b-76117c-7b6dc4bd8-tjgr5 Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd Stopping container payload-pre-processing 25m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Startup probe error: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused... 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-69c573-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw Startup probe failed: Get "http://10.134.0.216:15021/healthz/ready": dial tcp 10.134.0.216:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Readiness probe failed: Get "http://10.134.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-69c573-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw Startup probe error: Get "http://10.134.0.216:15021/healthz/ready": dial tcp 10.134.0.216:15021: connect: connection refused... 25m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Readiness probe error: Get "http://10.134.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Warning ProbeError pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Startup probe error: Get "http://10.132.0.199:15021/healthz/ready": dial tcp 10.132.0.199:15021: connect: connection refused... 25m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Container started 25m Warning Unhealthy pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Startup probe failed: Get "http://10.132.0.199:15021/healthz/ready": dial tcp 10.132.0.199:15021: connect: connection refused 25m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Container created 25m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Startup probe error: Get "http://10.134.0.217:15021/healthz/ready": dial tcp 10.134.0.217:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Startup probe failed: Get "http://10.134.0.217:15021/healthz/ready": dial tcp 10.134.0.217:15021: connect: connection refused 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-1e7123-openshift-default Scaled up replica set e2e-shared-b-w0-1e7123-openshift-default-796568db5f from 1 to 2 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-1e7123-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-1e7123-openshift-default-796568db5f Created pod: e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 25m Normal Pulled pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 Container created 25m Normal Started pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 Container started 25m Normal AddedInterface pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 Add eth0 [10.134.0.218/23] from ovn-kubernetes 24m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Startup probe failed: Get "http://10.134.0.203:15021/healthz/ready": dial tcp 10.134.0.203:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Startup probe error: Get "http://10.134.0.203:15021/healthz/ready": dial tcp 10.134.0.203:15021: connect: connection refused... 24m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz Container created 24m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz Add eth0 [10.132.0.200/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz Container started 24m Warning ProbeError pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 Startup probe error: Get "http://10.134.0.218:15021/healthz/ready": dial tcp 10.134.0.218:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 Startup probe failed: Get "http://10.134.0.218:15021/healthz/ready": dial tcp 10.134.0.218:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz Startup probe error: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz Startup probe failed: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-69c573-openshift-default New size: 10; reason: 24m Normal Killing pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Stopping container istio-proxy 24m Normal Killing pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-68b4w Stopping container istio-proxy 24m Normal Killing pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Stopping container istio-proxy 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-69c573-openshift-default-95546bd69 (combined from similar events): Created pod: e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw 24m Normal Killing pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Stopping container istio-proxy 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-69c573-openshift-default Scaled up replica set e2e-shared-a-w6-69c573-openshift-default-95546bd69 from 8 to 10 24m Warning Unhealthy pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning Unhealthy pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-178bcc-openshift-default Scaled up replica set e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 from 0 to 1 24m Warning ProbeError pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-c8tp4 Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal AddedLabel gateway/e2e-shared-a-w5-178bcc Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-178bcc 24m Warning ProbeError pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 Created pod: e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx 24m Warning ProbeError pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-rk8c8 Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt Add eth0 [10.133.0.162/23] from ovn-kubernetes 24m Warning ProbeError pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Readiness probe error: Get "http://10.134.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Readiness probe error: Get "http://10.134.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-76117c-5f9c6f6859-6f7cd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Readiness probe failed: Get "http://10.134.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Readiness probe failed: Get "http://10.134.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt Container started 24m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt Container created 24m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c Add eth0 [10.132.0.201/23] from ovn-kubernetes 24m Normal Created pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c Container created 24m Normal Started pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw Container started 24m Normal Pulled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw Container created 24m Normal Pulled pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c Container started 24m Normal AddedInterface pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw Add eth0 [10.132.0.202/23] from ovn-kubernetes 24m Normal AddedInterface pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Add eth0 [10.134.0.219/23] from ovn-kubernetes 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-1b1531-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-1b1531-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74 Created pod: payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-178bcc Scaled up replica set payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499 Created pod: payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-pc9sr 24m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt Startup probe failed: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-178bcc Scaled up replica set payload-processing-e2e-shared-a-w5-178bcc-58cc896d74 from 0 to 1 24m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt Startup probe error: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused... 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-178bcc: serviceaccount "payload-processing-e2e-shared-a-w5-178bcc" not found 24m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74 Error creating: pods "payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-178bcc: serviceaccount "payload-processing-e2e-shared-a-w5-178bcc" not found 24m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-pc9sr Container started 24m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c Startup probe failed: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused 24m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw Startup probe failed: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused 24m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-pc9sr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-pc9sr Container created 24m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c Startup probe error: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused... 24m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-pc9sr Add eth0 [10.134.0.221/23] from ovn-kubernetes 24m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw Startup probe error: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused... 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf Created pod: e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0a6e66-openshift-default Scaled up replica set e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf from 4 to 6 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-0a6e66-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf Created pod: e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-1b1531-openshift-default New size: 8; reason: 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-1b1531-openshift-default Scaled up replica set e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 from 4 to 8 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-e3586f-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e3586f-openshift-default Scaled up replica set e2e-shared-b-w1-e3586f-openshift-default-976787cfc from 4 to 5 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-e3586f-openshift-default-976787cfc Created pod: e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg 24m Normal ScalingReplicaSet deployment/e2e-inf-b-1fa500-openshift-default Scaled up replica set e2e-inf-b-1fa500-openshift-default-5f684fd55f from 2 to 3 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-1fa500-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 24m Normal SuccessfulCreate replicaset/e2e-inf-b-1fa500-openshift-default-5f684fd55f Created pod: e2e-inf-b-1fa500-openshift-default-5f684fd55f-x4nvl 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-618d93-openshift-default-79769f779 Created pod: e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-618d93-openshift-default Scaled up replica set e2e-shared-a-w1-618d93-openshift-default-79769f779 from 2 to 3 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-618d93-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-1b1531-openshift-default Scaled up replica set e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 from 8 to 10 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-1b1531-openshift-default New size: 10; reason: All metrics below target 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 (combined from similar events): Created pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-178bcc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-178bcc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz Stopping container payload-pre-processing 23m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-inf-a-f2dbd9 Failed to update endpoint openshift-ingress/payload-processing-e2e-inf-a-f2dbd9: Operation cannot be fulfilled on endpoints "payload-processing-e2e-inf-a-f2dbd9": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-inf-a-f2dbd9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 7eeb85c6-efd2-49a1-a0c2-838676c531ae, UID in object meta: 23m Normal Killing pod/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb Stopping container payload-processing 23m Warning Unhealthy pod/payload-processing-e2e-inf-a-f2dbd9-7b9996c75d-gnthb Readiness probe failed: timeout: failed to connect service "10.132.0.159:50052" within 1s: context deadline exceeded 23m Normal AddedInterface pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 Add eth0 [10.132.0.205/23] from ovn-kubernetes 23m Normal Created pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 Container created 23m Normal Created pod/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr Container created 23m Normal Pulled pod/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Started pod/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr Container started 23m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr Add eth0 [10.132.0.204/23] from ovn-kubernetes 23m Normal Pulled pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 Container started 23m Normal AddedLabel gateway/e2e-ipp-a-7e6199 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-7e6199 23m Normal ScalingReplicaSet deployment/e2e-ipp-a-7e6199-openshift-default Scaled up replica set e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5 from 0 to 1 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5 Created pod: e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 23m Normal Killing pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-8wp8d Stopping container istio-proxy 23m Normal Killing pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Stopping container istio-proxy 23m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-f2dbd9-56d55cc99f-jhxpz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning ProbeError pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-inf-a-f2dbd9-openshift-default-776ff48945-s29qr Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Readiness probe error: Get "http://10.134.0.211:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h Readiness probe failed: timeout: failed to connect service "10.132.0.120:50052" within 1s: context deadline exceeded 23m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Readiness probe failed: Get "http://10.134.0.211:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Warning Unhealthy pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 Startup probe failed: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 Startup probe error: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused... 23m Normal AddedInterface pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 Add eth0 [10.132.0.208/23] from ovn-kubernetes 23m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 Add eth0 [10.134.0.230/23] from ovn-kubernetes 23m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 4; reason: All metrics below target 23m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 Container started 23m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default Scaled down replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 5 to 4 23m Normal SuccessfulDelete replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Deleted pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx 23m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 Stopping container istio-proxy 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 4; reason: All metrics below target 23m Normal SuccessfulDelete replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Deleted pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 23m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default Scaled down replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 5 to 4 23m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-zlrjx Stopping container istio-proxy 23m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 Container created 23m Normal Killing pod/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp Stopping container payload-pre-processing 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79 Error creating: pods "payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7e6199: serviceaccount "payload-processing-e2e-ipp-a-7e6199" not found 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-7e6199 Scaled up replica set payload-processing-e2e-ipp-a-7e6199-6ff9868fc4 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4 Created pod: payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79 Created pod: payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-chtgk 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4 Error creating: pods "payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7e6199: serviceaccount "payload-processing-e2e-ipp-a-7e6199" not found 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-7e6199 Scaled up replica set payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79 from 0 to 1 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-chtgk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-chtgk Add eth0 [10.132.0.209/23] from ovn-kubernetes 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-chtgk Container started 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-chtgk Container created 23m Normal Created pod/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n Container created 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Created pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv 23m Normal Started pod/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n Container started 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Created pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj 23m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-2405fb-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 23m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Add eth0 [10.134.0.231/23] from ovn-kubernetes 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n Add eth0 [10.133.0.164/23] from ovn-kubernetes 23m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 Startup probe failed: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused 23m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-2405fb-openshift-default Scaled up replica set e2e-shared-a-w0-2405fb-openshift-default-556d785b45 from 2 to 4 23m Normal Pulled pod/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 Startup probe error: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused... 23m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Container started 23m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Container created 23m Normal Pulled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Add eth0 [10.133.0.165/23] from ovn-kubernetes 23m Normal Started pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Container started 23m Normal Created pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Container created 23m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Startup probe error: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Startup probe failed: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-27b5d5-openshift-default New size: 2; reason: All metrics below target 23m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Startup probe failed: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused 23m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r Stopping container istio-proxy 23m Normal SuccessfulDelete replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Deleted pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-vbs5r 23m Normal SuccessfulDelete replicaset/e2e-worker-w4-27b5d5-openshift-default-66b797556c Deleted pod: e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr 23m Normal ScalingReplicaSet deployment/e2e-worker-w4-27b5d5-openshift-default Scaled down replica set e2e-worker-w4-27b5d5-openshift-default-66b797556c from 3 to 2 23m Normal Killing pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr Stopping container istio-proxy 23m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Startup probe error: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused... 23m Normal Killing pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Stopping container istio-proxy 23m Normal Killing pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-ndcjq Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning ProbeError pod/e2e-inf-b-1fa500-openshift-default-5f684fd55f-h7zzz Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/payload-processing-e2e-inf-b-1fa500-6cb9b7dc56-2lh5f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-1fa500-7ff4f97659-9mwbp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal AddedInterface pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Add eth0 [10.133.0.166/23] from ovn-kubernetes 22m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Container started 22m Normal Pulled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Container created 22m Normal Started pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Container started 22m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Add eth0 [10.132.0.213/23] from ovn-kubernetes 22m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Container created 22m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Container started 22m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Container created 22m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Add eth0 [10.132.0.214/23] from ovn-kubernetes 22m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Startup probe error: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Startup probe failed: Get "http://10.132.0.213:15021/healthz/ready": dial tcp 10.132.0.213:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Startup probe error: Get "http://10.132.0.213:15021/healthz/ready": dial tcp 10.132.0.213:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Startup probe failed: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-7e6199-5fccfbbd79-chtgk Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n Stopping container payload-processing 22m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Readiness probe failed: Get "http://10.132.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Readiness probe error: Get "http://10.132.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 Container started 22m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 Container created 22m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Warning Unhealthy pod/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n Readiness probe failed: timeout: failed to connect service "10.133.0.164:50052" within 1s: context deadline exceeded 22m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 Add eth0 [10.133.0.167/23] from ovn-kubernetes 22m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 Startup probe failed: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 Startup probe error: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused... 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff Created pod: e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 22m Normal AddedLabel gateway/e2e-ipp-a-f5613a Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f5613a 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-f5613a-openshift-default Scaled up replica set e2e-ipp-a-f5613a-openshift-default-55d9bc5bff from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-7e6199 Scaled up replica set payload-processing-e2e-ipp-a-7e6199-655bb49f4d from 0 to 1 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7 Error creating: pods "payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7e6199: serviceaccount "payload-processing-e2e-ipp-a-7e6199" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-7e6199 Scaled up replica set payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7 Created pod: payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-7e6199-655bb49f4d Created pod: payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-7e6199-655bb49f4d Error creating: pods "payload-processing-e2e-ipp-a-7e6199-655bb49f4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7e6199: serviceaccount "payload-processing-e2e-ipp-a-7e6199" not found 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss Add eth0 [10.134.0.235/23] from ovn-kubernetes 22m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.79s (1.79s including waiting). Image size: 166612641 bytes. 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss Container started 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Created pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-2405fb-openshift-default Scaled up replica set e2e-shared-a-w0-2405fb-openshift-default-556d785b45 from 4 to 7 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-2405fb-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Created pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Created pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-ddb6ea-openshift-default-c489869 Created pod: e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-ddb6ea-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-ddb6ea-openshift-default Scaled up replica set e2e-shared-a-w3-ddb6ea-openshift-default-c489869 from 2 to 3 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 21m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-855f9574cf Deleted pod: maas-default-gateway-openshift-default-855f9574cf-chjvh 21m Normal Killing pod/maas-default-gateway-openshift-default-855f9574cf-chjvh Stopping container istio-proxy 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-855f9574cf from 9 to 8 21m Normal ScalingReplicaSet deployment/e2e-ipp-a-e50648-openshift-default Scaled up replica set e2e-ipp-a-e50648-openshift-default-5b7dcc895f from 0 to 1 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-e50648-openshift-default-5b7dcc895f Created pod: e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg 21m Normal AddedLabel gateway/e2e-ipp-a-e50648 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-e50648 21m Normal AddedInterface pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 Add eth0 [10.134.0.237/23] from ovn-kubernetes 21m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Readiness probe error: Get "http://10.134.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Readiness probe failed: Get "http://10.134.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7 Error creating: pods "payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f5613a: serviceaccount "payload-processing-e2e-ipp-a-f5613a" not found 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f5613a-75fbd77c88 Created pod: payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f5613a-75fbd77c88 Error creating: pods "payload-processing-e2e-ipp-a-f5613a-75fbd77c88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f5613a: serviceaccount "payload-processing-e2e-ipp-a-f5613a" not found 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7 Created pod: payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-k5stc 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f5613a Scaled up replica set payload-processing-e2e-ipp-a-f5613a-75fbd77c88 from 0 to 1 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f5613a Scaled up replica set payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7 from 0 to 1 21m Normal SuccessfulDelete replicaset/e2e-worker-w5-2d5d53-openshift-default-6dd7664849 Deleted pod: e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s 21m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc Stopping container istio-proxy 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-k5stc Container started 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-k5stc Container created 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-k5stc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-k5stc Add eth0 [10.134.0.238/23] from ovn-kubernetes 21m Normal SuccessfulDelete replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Deleted pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc 21m Normal SuccessfulDelete replicaset/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f Deleted pod: e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ad23c7-openshift-default New size: 3; reason: All metrics below target 21m Normal ScalingReplicaSet deployment/e2e-worker-w0-ad23c7-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f from 4 to 3 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-2d5d53-openshift-default New size: 3; reason: All metrics below target 21m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-9mtsv Stopping container istio-proxy 21m Normal ScalingReplicaSet deployment/e2e-worker-w5-2d5d53-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-2d5d53-openshift-default-6dd7664849 from 4 to 3 21m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2tc5s Stopping container istio-proxy 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 Add eth0 [10.134.0.239/23] from ovn-kubernetes 21m Normal Started pod/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 Container started 21m Normal Created pod/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 Container created 21m Normal Pulled pod/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg Add eth0 [10.133.0.168/23] from ovn-kubernetes 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm Add eth0 [10.132.0.219/23] from ovn-kubernetes 21m Normal Pulling pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e50648 Scaled up replica set payload-processing-e2e-ipp-a-e50648-85599ff484 from 0 to 1 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e50648-85599ff484 Error creating: pods "payload-processing-e2e-ipp-a-e50648-85599ff484-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e50648: serviceaccount "payload-processing-e2e-ipp-a-e50648" not found 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e50648 Scaled up replica set payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87 from 0 to 1 21m Normal Pulled pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.703s (2.703s including waiting). Image size: 166612641 bytes. 21m Normal Started pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm Container started 21m Normal Created pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm Container created 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87 Created pod: payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87 Error creating: pods "payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e50648: serviceaccount "payload-processing-e2e-ipp-a-e50648" not found 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e50648-85599ff484-jnxb9 Add eth0 [10.134.0.240/23] from ovn-kubernetes 21m Normal Started pod/payload-processing-e2e-ipp-a-e50648-85599ff484-jnxb9 Container started 21m Normal Pulled pod/payload-processing-e2e-ipp-a-e50648-85599ff484-jnxb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e50648-85599ff484 Created pod: payload-processing-e2e-ipp-a-e50648-85599ff484-jnxb9 21m Normal Created pod/payload-processing-e2e-ipp-a-e50648-85599ff484-jnxb9 Container created 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 Add eth0 [10.132.0.221/23] from ovn-kubernetes 21m Normal Started pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 Container started 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 Container created 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 Container started 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 Container created 21m Normal Pulled pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n Container created 21m Normal Pulled pod/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n Container started 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f5613a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f5613a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-7e6199-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-7e6199-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning Unhealthy pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 Startup probe failed: Get "http://10.132.0.208:15021/healthz/ready": dial tcp 10.132.0.208:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 Startup probe error: Get "http://10.132.0.208:15021/healthz/ready": dial tcp 10.132.0.208:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 21m Warning ProbeError pod/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f Created pod: e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 21m Normal ScalingReplicaSet deployment/e2e-ipp-b-6d9f17-openshift-default Scaled up replica set e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f from 0 to 1 21m Normal AddedLabel gateway/e2e-ipp-b-6d9f17 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-6d9f17 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-8f3ff4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-8f3ff4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-7e6199-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-7e6199-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Created pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg Container created 20m Normal Started pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg Container started 20m Normal Pulled pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Warning ProbeError pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg Startup probe error: Get "http://10.133.0.168:15021/healthz/ready": dial tcp 10.133.0.168:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg Startup probe failed: Get "http://10.133.0.168:15021/healthz/ready": dial tcp 10.133.0.168:15021: connect: connection refused 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e50648-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e50648-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Killing pod/payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s Stopping container payload-processing 20m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-shared-a-w6-69c573 Failed to update endpoint openshift-ingress/payload-processing-e2e-shared-a-w6-69c573: Operation cannot be fulfilled on endpoints "payload-processing-e2e-shared-a-w6-69c573": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-shared-a-w6-69c573, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 208e80f1-0f70-4a49-968c-6869c33c9073, UID in object meta: 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt Stopping container payload-pre-processing 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e50648-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e50648-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-69c573-868f49556c-xjzkt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-69c573-689db57bf8-9hf4s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Stopping container istio-proxy 19m Warning ProbeError pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-w57dm Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-ns2qd Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-mpf6v Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-49cqw Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-j48lt Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-zxbbn Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal Killing pod/e2e-shared-a-w6-69c573-openshift-default-95546bd69-gh54c Stopping container istio-proxy 19m Normal Created pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg Container created 19m Normal AddedInterface pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Add eth0 [10.134.0.243/23] from ovn-kubernetes 19m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 Container created 19m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 Container started 19m Normal Created pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Container created 19m Normal Started pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Container started 19m Normal AddedInterface pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Add eth0 [10.132.0.233/23] from ovn-kubernetes 19m Normal Pulled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Pulled pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Killing pod/payload-processing-e2e-ipp-a-e50648-85599ff484-jnxb9 Stopping container payload-processing 19m Normal Created pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Container created 19m Normal Created pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Container created 19m Normal Started pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg Container started 19m Normal Pulled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Pulled pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg Add eth0 [10.134.0.245/23] from ovn-kubernetes 19m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 Add eth0 [10.134.0.244/23] from ovn-kubernetes 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 Stopping container payload-pre-processing 19m Normal Started pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Container started 19m Normal Started pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Container started 19m Normal Started pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm Container started 19m Normal Created pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm Container created 19m Normal AddedInterface pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Add eth0 [10.132.0.235/23] from ovn-kubernetes 19m Normal AddedInterface pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm Add eth0 [10.132.0.232/23] from ovn-kubernetes 19m Normal Pulled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Killing pod/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt Stopping container payload-pre-processing 19m Normal AddedLabel gateway/e2e-ipp-a-7aa217 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-7aa217 19m Normal ScalingReplicaSet deployment/e2e-ipp-a-7aa217-openshift-default Scaled up replica set e2e-ipp-a-7aa217-openshift-default-777b88d9b9 from 0 to 1 19m Normal SuccessfulCreate replicaset/e2e-ipp-a-7aa217-openshift-default-777b88d9b9 Created pod: e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql 19m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Startup probe failed: Get "http://10.134.0.243:15021/healthz/ready": dial tcp 10.134.0.243:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm Startup probe error: Get "http://10.132.0.232:15021/healthz/ready": dial tcp 10.132.0.232:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm Startup probe failed: Get "http://10.132.0.232:15021/healthz/ready": dial tcp 10.132.0.232:15021: connect: connection refused 19m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-a-e50648-6bcb65d87-x4jd4 Readiness probe failed: timeout: failed to connect service "10.132.0.221:50052" within 1s: context deadline exceeded 19m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Startup probe error: Get "http://10.134.0.243:15021/healthz/ready": dial tcp 10.134.0.243:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Startup probe failed: Get "http://10.132.0.233:15021/healthz/ready": dial tcp 10.132.0.233:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg Startup probe error: Get "http://10.134.0.245:15021/healthz/ready": dial tcp 10.134.0.245:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Startup probe failed: Get "http://10.132.0.235:15021/healthz/ready": dial tcp 10.132.0.235:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 Startup probe error: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused... 19m Normal AddedInterface pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 Add eth0 [10.134.0.246/23] from ovn-kubernetes 19m Normal Pulled pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 Container created 19m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 Startup probe failed: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Startup probe error: Get "http://10.132.0.235:15021/healthz/ready": dial tcp 10.132.0.235:15021: connect: connection refused... 19m Warning ProbeError pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Startup probe error: Get "http://10.132.0.233:15021/healthz/ready": dial tcp 10.132.0.233:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg Startup probe failed: Get "http://10.134.0.245:15021/healthz/ready": dial tcp 10.134.0.245:15021: connect: connection refused 19m Normal Started pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 Container started 19m Normal AddedInterface pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql Add eth0 [10.133.0.170/23] from ovn-kubernetes 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6 Created pod: payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455 Error creating: pods "payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e50648: serviceaccount "payload-processing-e2e-ipp-a-e50648" not found 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7 Created pod: payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-jphs8 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6 Error creating: pods "payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7aa217: serviceaccount "payload-processing-e2e-ipp-a-7aa217" not found 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e50648 Scaled up replica set payload-processing-e2e-ipp-a-e50648-549c7ddbb6 from 0 to 1 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-7aa217 Scaled up replica set payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6 from 0 to 1 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e50648-549c7ddbb6 Created pod: payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e50648-549c7ddbb6 Error creating: pods "payload-processing-e2e-ipp-a-e50648-549c7ddbb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e50648: serviceaccount "payload-processing-e2e-ipp-a-e50648" not found 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-7aa217 Scaled up replica set payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7 from 0 to 1 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7 Error creating: pods "payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7aa217: serviceaccount "payload-processing-e2e-ipp-a-7aa217" not found 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e50648 Scaled up replica set payload-pre-processing-e2e-ipp-a-e50648-6d6956b455 from 0 to 1 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-jphs8 Add eth0 [10.132.0.237/23] from ovn-kubernetes 19m Normal Created pod/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-jphs8 Container created 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455 Created pod: payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-jphs8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal Started pod/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-jphs8 Container started 19m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 Startup probe failed: Get "http://10.134.0.246:15021/healthz/ready": dial tcp 10.134.0.246:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 Startup probe error: Get "http://10.134.0.246:15021/healthz/ready": dial tcp 10.134.0.246:15021: connect: connection refused... 19m Normal Killing pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-dh9vq Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-mbxl6 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-wgg57 Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-0a6e66-74cdb87c47-dp4zt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-5xrv6 Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-0a6e66-57d9dc94-xl4p2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-sbmhk Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Pulled pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj Container created 19m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l Add eth0 [10.134.0.249/23] from ovn-kubernetes 19m Normal Created pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Container created 19m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l Container created 19m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l Container started 19m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 Add eth0 [10.134.0.250/23] from ovn-kubernetes 19m Normal AddedInterface pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Add eth0 [10.134.0.248/23] from ovn-kubernetes 19m Normal Started pod/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj Container started 19m Normal AddedInterface pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Add eth0 [10.132.0.240/23] from ovn-kubernetes 19m Normal Pulled pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Container created 19m Normal Started pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Container started 19m Normal Started pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Container started 19m Normal Pulled pod/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 Add eth0 [10.132.0.239/23] from ovn-kubernetes 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 Add eth0 [10.133.0.172/23] from ovn-kubernetes 19m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 19m Normal Pulled pod/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Normal Pulled pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 Container created 19m Normal AddedInterface pod/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj Add eth0 [10.133.0.171/23] from ovn-kubernetes 19m Normal Created pod/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 Container created 19m Normal Started pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 Container started 19m Normal Started pod/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 Container started 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.021s (2.021s including waiting). Image size: 166612641 bytes. 19m Normal Created pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 Container created 19m Normal Started pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 Container started 19m Warning ProbeError pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Readiness probe error: Get "http://10.134.0.219:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 19m Warning Unhealthy pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Readiness probe failed: Get "http://10.134.0.219:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 19m Normal Killing pod/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 Stopping container payload-processing 19m Warning Unhealthy pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Startup probe failed: Get "http://10.132.0.240:15021/healthz/ready": dial tcp 10.132.0.240:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Startup probe error: Get "http://10.132.0.240:15021/healthz/ready": dial tcp 10.132.0.240:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Startup probe failed: Get "http://10.134.0.248:15021/healthz/ready": dial tcp 10.134.0.248:15021: connect: connection refused 19m Normal Created pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Container created 19m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l Startup probe error: Get "http://10.134.0.249:15021/healthz/ready": dial tcp 10.134.0.249:15021: connect: connection refused... 19m Normal Pulled pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l Startup probe failed: Get "http://10.134.0.249:15021/healthz/ready": dial tcp 10.134.0.249:15021: connect: connection refused 19m Normal AddedLabel gateway/e2e-ipp-a-826518 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-826518 19m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Startup probe error: Get "http://10.134.0.248:15021/healthz/ready": dial tcp 10.134.0.248:15021: connect: connection refused... 19m Normal ScalingReplicaSet deployment/e2e-ipp-a-826518-openshift-default Scaled up replica set e2e-ipp-a-826518-openshift-default-67bd854bff from 0 to 1 19m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 Startup probe failed: Get "http://10.134.0.250:15021/healthz/ready": dial tcp 10.134.0.250:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 Startup probe error: Get "http://10.134.0.250:15021/healthz/ready": dial tcp 10.134.0.250:15021: connect: connection refused... 19m Normal SuccessfulCreate replicaset/e2e-ipp-a-826518-openshift-default-67bd854bff Created pod: e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr 19m Normal Started pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Container started 19m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 Created pod: e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-c2jdw 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-178bcc-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 19m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 Created pod: e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-67f4m 19m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 Created pod: e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-rpbzh 19m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-178bcc-openshift-default Scaled up replica set e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 from 1 to 4 18m Warning Unhealthy pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Startup probe failed: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused 18m Warning ProbeError pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Startup probe error: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused... 18m Normal ScalingReplicaSet deployment/e2e-ipp-b-1a844e-openshift-default Scaled up replica set e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 from 0 to 1 18m Normal AddedLabel gateway/e2e-ipp-b-1a844e Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-1a844e 18m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-178bcc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-178bcc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-worker-w6-bce71f-7df86d777d-qjwv9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-processing-e2e-shared-a-w5-178bcc-58cc896d74-cdxgr Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-178bcc-b4dc5499-pc9sr Stopping container payload-pre-processing 18m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-shared-a-w5-178bcc Failed to update endpoint openshift-ingress/payload-processing-e2e-shared-a-w5-178bcc: Operation cannot be fulfilled on endpoints "payload-processing-e2e-shared-a-w5-178bcc": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-shared-a-w5-178bcc, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 858a6880-4949-4190-b90a-49d9325b528e, UID in object meta: 18m Normal Pulled pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Started pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql Container started 18m Normal Created pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql Container created 18m Normal Started pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Container started 18m Normal Created pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Container created 18m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Add eth0 [10.134.0.253/23] from ovn-kubernetes 18m Normal Pulled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Add eth0 [10.132.0.243/23] from ovn-kubernetes 18m Normal Started pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Container started 18m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 Created pod: e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-9nptk 18m Normal Created pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Container created 18m Normal Pulled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Add eth0 [10.132.0.244/23] from ovn-kubernetes 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-178bcc-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 18m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 Created pod: e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-7ckdm 18m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-178bcc-openshift-default Scaled up replica set e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 from 4 to 8 18m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 Created pod: e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-dtvch 18m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9 Created pod: e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-g89kd 18m Normal Killing pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Stopping container istio-proxy 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1a844e Scaled up replica set payload-processing-e2e-ipp-b-1a844e-566cccccb8 from 0 to 1 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55 Error creating: pods "payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a844e: serviceaccount "payload-processing-e2e-ipp-b-1a844e" not found 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1a844e-566cccccb8 Error creating: pods "payload-processing-e2e-ipp-b-1a844e-566cccccb8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a844e: serviceaccount "payload-processing-e2e-ipp-b-1a844e" not found 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1a844e-566cccccb8 Created pod: payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1a844e Scaled up replica set payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55 from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55 Created pod: payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-k5s8s 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn Add eth0 [10.133.0.173/23] from ovn-kubernetes 18m Normal Started pod/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn Container started 18m Normal Pulled pod/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Warning Unhealthy pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql Startup probe failed: Get "http://10.133.0.170:15021/healthz/ready": dial tcp 10.133.0.170:15021: connect: connection refused 18m Normal Created pod/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn Container created 18m Warning ProbeError pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql Startup probe error: Get "http://10.133.0.170:15021/healthz/ready": dial tcp 10.133.0.170:15021: connect: connection refused... 18m Warning ProbeError pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal SuccessfulCreate replicaset/e2e-ipp-a-c2b092-openshift-default-758cf88db7 Created pod: e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-k5s8s Add eth0 [10.132.0.245/23] from ovn-kubernetes 18m Normal ScalingReplicaSet deployment/e2e-ipp-a-c2b092-openshift-default Scaled up replica set e2e-ipp-a-c2b092-openshift-default-758cf88db7 from 0 to 1 18m Normal AddedLabel gateway/e2e-ipp-a-c2b092 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-c2b092 18m Warning Unhealthy pod/e2e-shared-a-w5-178bcc-openshift-default-86ddcdb5b9-h7swx Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Startup probe failed: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused 18m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Startup probe error: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused... 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-k5s8s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-k5s8s Container started 18m Normal AddedInterface pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 Add eth0 [10.132.0.246/23] from ovn-kubernetes 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-k5s8s Container created 18m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Startup probe failed: Get "http://10.132.0.244:15021/healthz/ready": dial tcp 10.132.0.244:15021: connect: connection refused 18m Normal Pulled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Started pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 Container started 18m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Startup probe error: Get "http://10.132.0.244:15021/healthz/ready": dial tcp 10.132.0.244:15021: connect: connection refused... 18m Normal AddedInterface pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Add eth0 [10.134.0.254/23] from ovn-kubernetes 18m Normal Created pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 Container created 18m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txb2t Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-hwm7b Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-drsfx Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 Stopping container istio-proxy 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-6d9f17-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd Created pod: payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf Error creating: pods "payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c2b092: serviceaccount "payload-processing-e2e-ipp-a-c2b092" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf Created pod: payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c2b092 Scaled up replica set payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf from 0 to 1 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd Error creating: pods "payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c2b092: serviceaccount "payload-processing-e2e-ipp-a-c2b092" not found 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c2b092 Scaled up replica set payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd from 0 to 1 18m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 Startup probe error: Get "http://10.132.0.246:15021/healthz/ready": dial tcp 10.132.0.246:15021: connect: connection refused... 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-7aa217-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-7aa217-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 Startup probe failed: Get "http://10.132.0.246:15021/healthz/ready": dial tcp 10.132.0.246:15021: connect: connection refused 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Pulled pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 Container created 18m Normal AddedInterface pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 Add eth0 [10.132.0.248/23] from ovn-kubernetes 18m Normal Started pod/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 Container started 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 Add eth0 [10.134.1.0/23] from ovn-kubernetes 18m Normal Started pod/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 Container started 18m Normal Pulled pod/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 Container created 18m Normal Started pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 Container started 18m Normal Created pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 Container created 18m Normal Started pod/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 Container started 18m Normal AddedInterface pod/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 Add eth0 [10.133.0.175/23] from ovn-kubernetes 18m Normal Created pod/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 Container created 18m Normal Pulled pod/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 Add eth0 [10.134.0.255/23] from ovn-kubernetes 18m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-855f9574cf Deleted pod: maas-default-gateway-openshift-default-855f9574cf-lm7vd 18m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-855f9574cf from 8 to 6 18m Normal Killing pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd Stopping container istio-proxy 18m Normal Killing pod/maas-default-gateway-openshift-default-855f9574cf-v9tpn Stopping container istio-proxy 18m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 18m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-855f9574cf Deleted pod: maas-default-gateway-openshift-default-855f9574cf-v9tpn 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c2b092-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c2b092-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 Startup probe failed: Get "http://10.132.0.248:15021/healthz/ready": dial tcp 10.132.0.248:15021: connect: connection refused 18m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 Startup probe error: Get "http://10.132.0.248:15021/healthz/ready": dial tcp 10.132.0.248:15021: connect: connection refused... 18m Normal Killing pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg Stopping container istio-proxy 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-7e6199-7f4dc89bd7-z65ss Stopping container payload-pre-processing 18m Warning Unhealthy pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm Readiness probe failed: dial tcp 10.132.0.219:9004: connect: connection refused 18m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-e3586f-openshift-default Scaled down replica set e2e-shared-b-w1-e3586f-openshift-default-976787cfc from 5 to 4 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-e3586f-openshift-default New size: 4; reason: All metrics below target 18m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-e3586f-openshift-default-976787cfc Deleted pod: e2e-shared-b-w1-e3586f-openshift-default-976787cfc-x8fbg 18m Normal Killing pod/payload-processing-e2e-ipp-a-7e6199-655bb49f4d-6pjkm Stopping container payload-processing 18m Normal AddedInterface pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Add eth0 [10.134.1.10/23] from ovn-kubernetes 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-6d9f17-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Started pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 Container started 18m Normal Created pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 Container created 18m Normal Pulled pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 Add eth0 [10.132.1.5/23] from ovn-kubernetes 18m Normal Killing pod/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-7aa217-65949fbbd7-jphs8 Stopping container payload-pre-processing 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-cfa4bb-openshift-default (combined from similar events): New size: 3; reason: All metrics below target 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-7aa217-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk Stopping container istio-proxy 18m Normal SuccessfulDelete replicaset/e2e-worker-w2-39b867-openshift-default-756f9d477f Deleted pod: e2e-worker-w2-39b867-openshift-default-756f9d477f-wg8mk 18m Normal ScalingReplicaSet deployment/e2e-worker-w2-39b867-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-39b867-openshift-default-756f9d477f from 4 to 3 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-39b867-openshift-default (combined from similar events): New size: 3; reason: All metrics below target 18m Normal ScalingReplicaSet deployment/e2e-worker-w1-cfa4bb-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-cfa4bb-openshift-default-75f585bdc from 4 to 3 18m Normal SuccessfulDelete replicaset/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc Deleted pod: e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-7aa217-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-826518-56b65445b4 Error creating: pods "payload-processing-e2e-ipp-a-826518-56b65445b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-826518: serviceaccount "payload-processing-e2e-ipp-a-826518" not found 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-826518 Scaled up replica set payload-pre-processing-e2e-ipp-a-826518-6996d58587 from 0 to 1 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-826518-56b65445b4 Created pod: payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-826518-6996d58587 Error creating: pods "payload-pre-processing-e2e-ipp-a-826518-6996d58587-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-826518: serviceaccount "payload-processing-e2e-ipp-a-826518" not found 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-826518-6996d58587 Created pod: payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-826518 Scaled up replica set payload-processing-e2e-ipp-a-826518-56b65445b4 from 0 to 1 17m Normal Killing pod/e2e-ipp-a-7e6199-openshift-default-5cdcfcd5d5-twjq8 Stopping container istio-proxy 17m Warning ProbeError pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-fx76m Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal Started pod/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh Container started 17m Normal Created pod/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh Container created 17m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 Startup probe failed: Get "http://10.132.1.5:15021/healthz/ready": dial tcp 10.132.1.5:15021: connect: connection refused 17m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 Startup probe error: Get "http://10.132.1.5:15021/healthz/ready": dial tcp 10.132.1.5:15021: connect: connection refused... 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh Add eth0 [10.133.0.179/23] from ovn-kubernetes 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb Add eth0 [10.134.1.12/23] from ovn-kubernetes 17m Normal Pulled pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal Created pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb Container created 17m Normal Started pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb Container started 17m Normal Killing pod/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842 Stopping container istio-proxy 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c Created pod: payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-rqhfz 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-7aa217 Scaled up replica set payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c from 0 to 1 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-7aa217-7bc848b57b Created pod: payload-processing-e2e-ipp-a-7aa217-7bc848b57b-psqwk 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-7aa217-7bc848b57b Error creating: pods "payload-processing-e2e-ipp-a-7aa217-7bc848b57b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7aa217: serviceaccount "payload-processing-e2e-ipp-a-7aa217" not found 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-7aa217 Scaled up replica set payload-processing-e2e-ipp-a-7aa217-7bc848b57b from 0 to 1 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c Error creating: pods "payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7aa217: serviceaccount "payload-processing-e2e-ipp-a-7aa217" not found 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1a844e-84f6d75f55-k5s8s Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d Stopping container payload-processing 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-rqhfz Add eth0 [10.132.1.8/23] from ovn-kubernetes 17m Normal Pulled pod/payload-processing-e2e-ipp-a-7aa217-7bc848b57b-psqwk Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-rqhfz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-processing-e2e-ipp-a-7aa217-7bc848b57b-psqwk Container started 17m Normal Created pod/payload-processing-e2e-ipp-a-7aa217-7bc848b57b-psqwk Container created 17m Normal Killing pod/payload-processing-e2e-ipp-a-f5613a-75fbd77c88-vvvk7 Stopping container payload-processing 17m Normal AddedInterface pod/payload-processing-e2e-ipp-a-7aa217-7bc848b57b-psqwk Add eth0 [10.132.1.9/23] from ovn-kubernetes 17m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-ipp-a-f5613a Failed to update endpoint openshift-ingress/payload-processing-e2e-ipp-a-f5613a: Operation cannot be fulfilled on endpoints "payload-processing-e2e-ipp-a-f5613a": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-ipp-a-f5613a, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 97c2736a-8007-43d6-b0f5-08571536ae10, UID in object meta: 17m Normal Created pod/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-rqhfz Container created 17m Normal Started pod/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-rqhfz Container started 17m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f5613a-8d74b8bd7-k5stc Stopping container payload-pre-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c2b092-584b67dfcf-k97v2 Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-ipp-a-c2b092-5cc45dc6bd-t67z2 Stopping container payload-processing 17m Normal Started pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 Container started 17m Normal Created pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 Container created 17m Normal Pulled pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Killing pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-kmlb5 Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-ffzcs Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w1-618d93-openshift-default-79769f779-bhvm9 Stopping container istio-proxy 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f5613a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Warning Unhealthy pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 Startup probe failed: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f5613a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Warning ProbeError pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 Startup probe error: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused... 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1a844e-59d5b45d66 Error creating: pods "payload-processing-e2e-ipp-b-1a844e-59d5b45d66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a844e: serviceaccount "payload-processing-e2e-ipp-b-1a844e" not found 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795 Error creating: pods "payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a844e: serviceaccount "payload-processing-e2e-ipp-b-1a844e" not found 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1a844e Scaled up replica set payload-processing-e2e-ipp-b-1a844e-59d5b45d66 from 0 to 1 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1a844e Scaled up replica set payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795 from 0 to 1 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1a844e-59d5b45d66 Created pod: payload-processing-e2e-ipp-b-1a844e-59d5b45d66-rjhs5 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795 Created pod: payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-t442z 17m Normal Created pod/payload-processing-e2e-ipp-b-1a844e-59d5b45d66-rjhs5 Container created 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1a844e-59d5b45d66-rjhs5 Add eth0 [10.134.1.14/23] from ovn-kubernetes 17m Normal Pulled pod/payload-processing-e2e-ipp-b-1a844e-59d5b45d66-rjhs5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-processing-e2e-ipp-b-1a844e-59d5b45d66-rjhs5 Container started 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-t442z Container started 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-t442z Container created 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-t442z Add eth0 [10.134.1.13/23] from ovn-kubernetes 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-t442z Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal SuccessfulCreate replicaset/e2e-ipp-a-c2b092-openshift-default-758cf88db7 Created pod: e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp 17m Normal SuccessfulCreate replicaset/e2e-ipp-a-c2b092-openshift-default-758cf88db7 Created pod: e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq 17m Normal ScalingReplicaSet deployment/e2e-ipp-a-c2b092-openshift-default Scaled up replica set e2e-ipp-a-c2b092-openshift-default-758cf88db7 from 1 to 4 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-c2b092-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 17m Normal Killing pod/e2e-ipp-a-f5613a-openshift-default-55d9bc5bff-wcb98 Stopping container istio-proxy 17m Normal SuccessfulCreate replicaset/e2e-ipp-a-c2b092-openshift-default-758cf88db7 Created pod: e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 17m Normal AddedInterface pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Add eth0 [10.133.0.181/23] from ovn-kubernetes 17m Normal AddedInterface pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp Add eth0 [10.132.1.11/23] from ovn-kubernetes 17m Normal AddedInterface pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq Add eth0 [10.132.1.12/23] from ovn-kubernetes 17m Normal Created pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Container created 17m Normal Created pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq Container created 17m Normal Started pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Container started 17m Normal Pulled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp Container started 17m Normal Created pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp Container created 17m Normal Pulled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq Container started 17m Normal Pulled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal SuccessfulCreate replicaset/e2e-ipp-b-aa49c5-openshift-default-6b56844555 Created pod: e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc 17m Normal ScalingReplicaSet deployment/e2e-ipp-b-aa49c5-openshift-default Scaled up replica set e2e-ipp-b-aa49c5-openshift-default-6b56844555 from 0 to 1 17m Normal AddedLabel gateway/e2e-ipp-b-aa49c5 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-aa49c5 17m Normal AddedInterface pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Add eth0 [10.132.1.13/23] from ovn-kubernetes 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c2b092 Scaled up replica set payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9 from 0 to 1 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c2b092 Scaled up replica set payload-processing-e2e-ipp-a-c2b092-54548c6cf4 from 0 to 1 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9 Error creating: pods "payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c2b092: serviceaccount "payload-processing-e2e-ipp-a-c2b092" not found 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9 Created pod: payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-zf2r5 17m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-shared-b-w1-e3586f Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w1-e3586f: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w1-e3586f": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w1-e3586f, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: bd5f601b-85ad-4968-98f1-3593afae29aa, UID in object meta: 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c2b092-54548c6cf4 Created pod: payload-processing-e2e-ipp-a-c2b092-54548c6cf4-2v2zd 17m Normal Killing pod/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 Stopping container payload-processing 17m Warning ProbeError pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Readiness probe error: Get "http://10.134.0.254:15021/healthz/ready": read tcp 10.134.0.2:39696->10.134.0.254:15021: read: connection reset by peer... 17m Warning Unhealthy pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Readiness probe failed: Get "http://10.134.0.254:15021/healthz/ready": read tcp 10.134.0.2:39696->10.134.0.254:15021: read: connection reset by peer 17m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 Stopping container payload-pre-processing 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c2b092-54548c6cf4 Error creating: pods "payload-processing-e2e-ipp-a-c2b092-54548c6cf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c2b092: serviceaccount "payload-processing-e2e-ipp-a-c2b092" not found 17m Normal Pulled pod/payload-processing-e2e-ipp-a-c2b092-54548c6cf4-2v2zd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Pulled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Created pod/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-zf2r5 Container created 17m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c2b092-54548c6cf4-2v2zd Add eth0 [10.132.1.15/23] from ovn-kubernetes 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-zf2r5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-zf2r5 Add eth0 [10.134.1.15/23] from ovn-kubernetes 17m Normal Started pod/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-zf2r5 Container started 17m Normal Created pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Container created 17m Normal Started pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Container started 17m Normal Killing pod/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh Stopping container payload-pre-processing 17m Normal Started pod/payload-processing-e2e-ipp-a-c2b092-54548c6cf4-2v2zd Container started 17m Normal Created pod/payload-processing-e2e-ipp-a-c2b092-54548c6cf4-2v2zd Container created 17m Normal Killing pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb Stopping container payload-processing 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d Readiness probe failed: timeout: failed to connect service "10.133.0.109:50052" within 1s: context deadline exceeded 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-aa49c5 Scaled up replica set payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb from 0 to 1 17m Warning ProbeError pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb Error creating: pods "payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-aa49c5: serviceaccount "payload-processing-e2e-ipp-b-aa49c5" not found 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-aa49c5 Scaled up replica set payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9 from 0 to 1 17m Warning ProbeError pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp Startup probe error: Get "http://10.132.1.11:15021/healthz/ready": dial tcp 10.132.1.11:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp Startup probe failed: Get "http://10.132.1.11:15021/healthz/ready": dial tcp 10.132.1.11:15021: connect: connection refused 17m Warning Unhealthy pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq Startup probe failed: Get "http://10.132.1.12:15021/healthz/ready": dial tcp 10.132.1.12:15021: connect: connection refused 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb Created pod: payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-htnjx 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9 Error creating: pods "payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-aa49c5: serviceaccount "payload-processing-e2e-ipp-b-aa49c5" not found 17m Warning ProbeError pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq Startup probe error: Get "http://10.132.1.12:15021/healthz/ready": dial tcp 10.132.1.12:15021: connect: connection refused... 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9 Created pod: payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-nt6th 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-htnjx Add eth0 [10.134.1.17/23] from ovn-kubernetes 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-nt6th Add eth0 [10.134.1.16/23] from ovn-kubernetes 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-nt6th Container created 17m Normal Created pod/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-htnjx Container created 17m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Readiness probe error: Get "http://10.134.0.253:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Readiness probe failed: Get "http://10.134.0.253:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Normal Pulled pod/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-htnjx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-htnjx Container started 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-nt6th Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-nt6th Container started 17m Normal Killing pod/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 Stopping container payload-processing 17m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-shared-a-w0-2405fb Failed to update endpoint openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb: Operation cannot be fulfilled on endpoints "payload-processing-e2e-shared-a-w0-2405fb": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 75829c45-fc7a-477c-8dba-ab551823c534, UID in object meta: 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk Stopping container payload-pre-processing 17m Warning Unhealthy pod/payload-processing-e2e-ipp-a-826518-56b65445b4-sf4rb Readiness probe failed: timeout: failed to connect service "10.134.1.12:50052" within 1s: context deadline exceeded 17m Warning ProbeError pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Startup probe error: Get "http://10.134.0.254:15021/healthz/ready": dial tcp 10.134.0.254:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Startup probe failed: Get "http://10.134.0.254:15021/healthz/ready": dial tcp 10.134.0.254:15021: connect: connection refused 17m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 Add eth0 [10.132.1.17/23] from ovn-kubernetes 17m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf Container started 17m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 Container created 17m Normal ScalingReplicaSet deployment/e2e-ipp-b-1a844e-openshift-default Scaled up replica set e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 from 1 to 4 17m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf 17m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf Container created 17m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 Container started 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-1a844e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 17m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 Add eth0 [10.133.0.182/23] from ovn-kubernetes 17m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf Add eth0 [10.134.1.18/23] from ovn-kubernetes 17m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 17m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 Container created 17m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 Container started 17m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 17m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Deleted pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 17m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Container started 17m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-2405fb-openshift-default-556d785b45 Deleted pod: e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm 17m Normal Killing pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-6x6mm Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-tjls7 Stopping container istio-proxy 17m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-2405fb-openshift-default Scaled down replica set e2e-shared-a-w0-2405fb-openshift-default-556d785b45 from 7 to 5 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-2405fb-openshift-default New size: 5; reason: All metrics below target 17m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Container created 17m Normal AddedLabel gateway/e2e-ipp-b-c0a27e Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-c0a27e 17m Normal SuccessfulCreate replicaset/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45 Created pod: e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km 17m Normal ScalingReplicaSet deployment/e2e-ipp-b-c0a27e-openshift-default Scaled up replica set e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45 from 0 to 1 17m Normal AddedInterface pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Add eth0 [10.133.0.183/23] from ovn-kubernetes 17m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-e3586f-77459df5d6-vq2z2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-e3586f-56ff5574c5-vljh7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-48cd4 Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-lt6t6 Stopping container istio-proxy 17m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf Startup probe error: Get "http://10.134.1.18:15021/healthz/ready": dial tcp 10.134.1.18:15021: connect: connection refused... 17m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 Startup probe error: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Startup probe failed: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused 17m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Startup probe error: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 Startup probe failed: Get "http://10.132.1.17:15021/healthz/ready": dial tcp 10.132.1.17:15021: connect: connection refused 17m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf Startup probe failed: Get "http://10.134.1.18:15021/healthz/ready": dial tcp 10.134.1.18:15021: connect: connection refused 17m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 Startup probe failed: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused 17m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 Startup probe error: Get "http://10.132.1.17:15021/healthz/ready": dial tcp 10.132.1.17:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-m7xzt Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-c0a27e Scaled up replica set payload-processing-e2e-ipp-b-c0a27e-7d57b54c98 from 0 to 1 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c Error creating: pods "payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c0a27e: serviceaccount "payload-processing-e2e-ipp-b-c0a27e" not found 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98 Error creating: pods "payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c0a27e: serviceaccount "payload-processing-e2e-ipp-b-c0a27e" not found 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98 Created pod: payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-j2hcl 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c Created pod: payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-vtdcl 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-c0a27e Scaled up replica set payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c from 0 to 1 17m Warning Unhealthy pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Readiness probe failed: Get "http://10.134.1.10:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-vtdcl Add eth0 [10.132.1.20/23] from ovn-kubernetes 17m Normal Created pod/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-j2hcl Container created 17m Warning ProbeError pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Readiness probe error: Get "http://10.134.1.10:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-vtdcl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-j2hcl Add eth0 [10.132.1.19/23] from ovn-kubernetes 17m Normal Pulled pod/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-j2hcl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-j2hcl Container started 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-vtdcl Container created 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-vtdcl Container started 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-826518 Scaled up replica set payload-processing-e2e-ipp-a-826518-7c7d6889c4 from 0 to 1 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-826518 Scaled up replica set payload-pre-processing-e2e-ipp-a-826518-5879559bc5 from 0 to 1 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-826518-7c7d6889c4 Error creating: pods "payload-processing-e2e-ipp-a-826518-7c7d6889c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-826518: serviceaccount "payload-processing-e2e-ipp-a-826518" not found 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-826518-5879559bc5 Error creating: pods "payload-pre-processing-e2e-ipp-a-826518-5879559bc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-826518: serviceaccount "payload-processing-e2e-ipp-a-826518" not found 16m Normal Started pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Container started 16m Normal Started pod/payload-processing-e2e-ipp-a-826518-7c7d6889c4-8gkdn Container started 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-826518-7c7d6889c4 Created pod: payload-processing-e2e-ipp-a-826518-7c7d6889c4-8gkdn 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Pulled pod/payload-processing-e2e-ipp-a-826518-7c7d6889c4-8gkdn Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/payload-processing-e2e-ipp-a-826518-7c7d6889c4-8gkdn Add eth0 [10.132.1.21/23] from ovn-kubernetes 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-826518-5879559bc5 Created pod: payload-pre-processing-e2e-ipp-a-826518-5879559bc5-tf7wf 16m Normal Created pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Container created 16m Normal Pulled pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/payload-processing-e2e-ipp-a-826518-7c7d6889c4-8gkdn Container created 16m Normal Created pod/payload-pre-processing-e2e-ipp-a-826518-5879559bc5-tf7wf Container created 16m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-826518-5879559bc5-tf7wf Add eth0 [10.134.1.19/23] from ovn-kubernetes 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-826518-5879559bc5-tf7wf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal Started pod/payload-pre-processing-e2e-ipp-a-826518-5879559bc5-tf7wf Container started 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c2b092-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c2b092-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-aa49c5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-aa49c5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-worker-w1-cfa4bb Failed to update endpoint openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb: Operation cannot be fulfilled on endpoints "payload-processing-e2e-worker-w1-cfa4bb": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: c013a51b-7d62-4130-9d6d-c4ba6b119af5, UID in object meta: 16m Normal Killing pod/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 Stopping container payload-processing 16m Warning ProbeError pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Startup probe error: Get "http://10.134.1.10:15021/healthz/ready": dial tcp 10.134.1.10:15021: connect: connection refused... 16m Warning Unhealthy pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Startup probe failed: Get "http://10.134.1.10:15021/healthz/ready": dial tcp 10.134.1.10:15021: connect: connection refused 16m Normal Killing pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-lx44v Stopping container istio-proxy 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-826518-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-826518-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk Readiness probe failed: timeout: failed to connect service "10.133.0.106:50052" within 1s: context deadline exceeded 16m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xjxgp Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-vx26d Readiness probe error: HTTP probe failed with statuscode: 503... 16m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2 Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-a-e50648-549c7ddbb6-67w85 Stopping container payload-processing 16m Normal AddedLabel gateway/e2e-ipp-b-be6b74 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-be6b74 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-c2b092-openshift-default New size: 5; reason: All metrics below target 16m Normal SuccessfulCreate replicaset/e2e-ipp-b-be6b74-openshift-default-954dcdd97 Created pod: e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k 16m Normal ScalingReplicaSet deployment/e2e-ipp-b-be6b74-openshift-default Scaled up replica set e2e-ipp-b-be6b74-openshift-default-954dcdd97 from 0 to 1 16m Normal SuccessfulCreate replicaset/e2e-ipp-a-c2b092-openshift-default-758cf88db7 Created pod: e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d 16m Normal ScalingReplicaSet deployment/e2e-ipp-a-c2b092-openshift-default Scaled up replica set e2e-ipp-a-c2b092-openshift-default-758cf88db7 from 4 to 5 16m Normal AddedInterface pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Add eth0 [10.132.1.23/23] from ovn-kubernetes 16m Normal Started pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d Container started 16m Normal Pulled pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d Add eth0 [10.134.1.20/23] from ovn-kubernetes 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1a844e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1a844e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal Created pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d Container created 16m Normal Killing pod/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz Stopping container payload-pre-processing 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-c0a27e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-c0a27e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning Unhealthy pod/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-be6b74-7c46964458 Error creating: pods "payload-processing-e2e-ipp-b-be6b74-7c46964458-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-be6b74: serviceaccount "payload-processing-e2e-ipp-b-be6b74" not found 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-be6b74 Scaled up replica set payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787 from 0 to 1 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-be6b74 Scaled up replica set payload-processing-e2e-ipp-b-be6b74-7c46964458 from 0 to 1 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787 Created pod: payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-wpmft 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787 Error creating: pods "payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-be6b74: serviceaccount "payload-processing-e2e-ipp-b-be6b74" not found 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-be6b74-7c46964458 Created pod: payload-processing-e2e-ipp-b-be6b74-7c46964458-qddjb 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-be6b74-7c46964458-qddjb Add eth0 [10.132.1.27/23] from ovn-kubernetes 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-wpmft Container created 16m Warning ProbeError pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Readiness probe error: Get "http://10.132.1.13:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Readiness probe failed: Get "http://10.132.1.13:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-wpmft Add eth0 [10.132.1.26/23] from ovn-kubernetes 16m Normal Pulled pod/payload-processing-e2e-ipp-b-be6b74-7c46964458-qddjb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-wpmft Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-wpmft Container started 16m Normal Started pod/payload-processing-e2e-ipp-b-be6b74-7c46964458-qddjb Container started 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-aa49c5-76f9d799c9-nt6th Stopping container payload-pre-processing 16m Normal Created pod/payload-processing-e2e-ipp-b-be6b74-7c46964458-qddjb Container created 16m Normal Killing pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg Stopping container istio-proxy 16m Normal Killing pod/payload-processing-e2e-ipp-b-aa49c5-6b8976b7cb-htnjx Stopping container payload-processing 16m Warning Unhealthy pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d Startup probe failed: Get "http://10.134.1.20:15021/healthz/ready": dial tcp 10.134.1.20:15021: connect: connection refused 16m Warning ProbeError pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d Startup probe error: Get "http://10.134.1.20:15021/healthz/ready": dial tcp 10.134.1.20:15021: connect: connection refused... 16m Normal Pulled pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Started pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Container started 16m Normal Created pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Container created 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-1a844e-openshift-default New size: 8; reason: All metrics below target 16m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr Add eth0 [10.132.1.30/23] from ovn-kubernetes 16m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 Add eth0 [10.133.0.186/23] from ovn-kubernetes 16m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr Add eth0 [10.134.1.21/23] from ovn-kubernetes 16m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr Container created 16m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr Container started 16m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z Add eth0 [10.132.1.29/23] from ovn-kubernetes 16m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z 16m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr 16m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 16m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr 16m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal ScalingReplicaSet deployment/e2e-ipp-b-1a844e-openshift-default Scaled up replica set e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 from 4 to 8 16m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr Container created 16m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z Container created 16m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr Container started 16m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 Container started 16m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z Container started 16m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 Container created 16m Warning Unhealthy pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Startup probe failed: Get "http://10.132.1.13:15021/healthz/ready": dial tcp 10.132.1.13:15021: connect: connection refused 16m Warning ProbeError pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Startup probe error: Get "http://10.132.1.13:15021/healthz/ready": dial tcp 10.132.1.13:15021: connect: connection refused... 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-q2k9x Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Stopping container istio-proxy 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Deleted pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 16m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-1b1531-openshift-default Scaled down replica set e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 from 10 to 7 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-1b1531-openshift-default New size: 7; reason: All metrics below target 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Deleted pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55 Deleted pod: e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 16m Normal Killing pod/payload-processing-e2e-ipp-b-c0a27e-7d57b54c98-j2hcl Stopping container payload-processing 16m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tx8s8 Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Stopping container istio-proxy 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-c0a27e-c76cbb78c-vtdcl Stopping container payload-pre-processing 16m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-6dsl7 Stopping container istio-proxy 16m Warning Unhealthy pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-b-w0-1e7123-openshift-default-796568db5f-l8497 Readiness probe error: HTTP probe failed with statuscode: 503... 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-aa49c5 Scaled up replica set payload-processing-e2e-ipp-b-aa49c5-c48688486 from 0 to 1 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-826518-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-aa49c5-c48688486 Created pod: payload-processing-e2e-ipp-b-aa49c5-c48688486-ljmsd 16m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr Startup probe error: Get "http://10.134.1.21:15021/healthz/ready": dial tcp 10.134.1.21:15021: connect: connection refused... 16m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 Startup probe failed: Get "http://10.133.0.186:15021/healthz/ready": dial tcp 10.133.0.186:15021: connect: connection refused 16m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-826518-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 Startup probe error: Get "http://10.133.0.186:15021/healthz/ready": dial tcp 10.133.0.186:15021: connect: connection refused... 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9 Error creating: pods "payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-aa49c5: serviceaccount "payload-processing-e2e-ipp-b-aa49c5" not found 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9 Created pod: payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-vcb79 16m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr Startup probe failed: Get "http://10.134.1.21:15021/healthz/ready": dial tcp 10.134.1.21:15021: connect: connection refused 16m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qtlz5 Readiness probe error: HTTP probe failed with statuscode: 503... 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-aa49c5 Scaled up replica set payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9 from 0 to 1 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-aa49c5-c48688486 Error creating: pods "payload-processing-e2e-ipp-b-aa49c5-c48688486-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-aa49c5: serviceaccount "payload-processing-e2e-ipp-b-aa49c5" not found 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-vcb79 Container created 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-vcb79 Add eth0 [10.134.1.23/23] from ovn-kubernetes 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-vcb79 Container started 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-vcb79 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal Started pod/payload-processing-e2e-ipp-b-aa49c5-c48688486-ljmsd Container started 16m Normal Created pod/payload-processing-e2e-ipp-b-aa49c5-c48688486-ljmsd Container created 16m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr Startup probe failed: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused 16m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z Startup probe error: Get "http://10.132.1.29:15021/healthz/ready": dial tcp 10.132.1.29:15021: connect: connection refused... 16m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z Startup probe failed: Get "http://10.132.1.29:15021/healthz/ready": dial tcp 10.132.1.29:15021: connect: connection refused 16m Normal Pulled pod/payload-processing-e2e-ipp-b-aa49c5-c48688486-ljmsd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr Startup probe error: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused... 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-aa49c5-c48688486-ljmsd Add eth0 [10.134.1.22/23] from ovn-kubernetes 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-1e7123-7688d76b9f-zn9kz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s Readiness probe failed: timeout: failed to connect service "10.133.0.114:50052" within 1s: context deadline exceeded 16m Normal Killing pod/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 Stopping container payload-processing 16m Normal Pulled pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Add eth0 [10.133.0.188/23] from ovn-kubernetes 16m Normal ScalingReplicaSet deployment/e2e-ipp-b-1a844e-openshift-default Scaled up replica set e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 from 8 to 9 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-1a844e-openshift-default New size: 9; reason: 16m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9 Created pod: e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-aa49c5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal Created pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Container created 16m Normal Killing pod/payload-processing-e2e-ipp-b-1a844e-59d5b45d66-rjhs5 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1a844e-7b9465b795-t442z Stopping container payload-pre-processing 16m Normal Started pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Container started 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-aa49c5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp Stopping container istio-proxy 16m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-h92cn Stopping container istio-proxy 16m Normal Killing pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rwpzl Stopping container istio-proxy 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d Created pod: payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-nxjsk 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9 Created pod: payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-tflx2 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-c0a27e Scaled up replica set payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9 from 0 to 1 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9 Error creating: pods "payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c0a27e: serviceaccount "payload-processing-e2e-ipp-b-c0a27e" not found 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d Error creating: pods "payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c0a27e: serviceaccount "payload-processing-e2e-ipp-b-c0a27e" not found 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-c0a27e Scaled up replica set payload-processing-e2e-ipp-b-c0a27e-74bbd9584d from 0 to 1 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-nxjsk Add eth0 [10.134.1.24/23] from ovn-kubernetes 16m Warning ProbeError pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Startup probe error: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused... 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-tflx2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-tflx2 Add eth0 [10.134.1.25/23] from ovn-kubernetes 16m Warning Unhealthy pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Startup probe failed: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused 16m Normal Started pod/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-nxjsk Container started 16m Normal Created pod/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-nxjsk Container created 16m Normal Pulled pod/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-nxjsk Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Warning ProbeError pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Readiness probe error: Get "http://10.133.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-tflx2 Container created 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-tflx2 Container started 16m Warning Unhealthy pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Readiness probe failed: Get "http://10.133.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-j5ljr Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-fv8pf Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-rz27z Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-64d84 Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hl9lr Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-kcs7x Stopping container istio-proxy 16m Normal Started pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Container started 16m Warning FailedToRetrieveImagePullSecret pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Unable to retrieve some image pull secrets (e2e-ipp-b-1a844e-openshift-default-dockercfg-465qc); attempting to pull the image may not succeed. 16m Normal Pulled pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Container created 16m Normal Killing pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz Stopping container istio-proxy 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-be6b74-6b5fb9f787-wpmft Stopping container payload-pre-processing 15m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-ipp-b-be6b74 Failed to update endpoint openshift-ingress/payload-processing-e2e-ipp-b-be6b74: Operation cannot be fulfilled on endpoints "payload-processing-e2e-ipp-b-be6b74": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-ipp-b-be6b74, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 099d2c3c-f21e-4fe9-9460-7ac66ee56057, UID in object meta: 15m Normal Killing pod/payload-processing-e2e-ipp-b-be6b74-7c46964458-qddjb Stopping container payload-processing 15m Warning Unhealthy pod/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Startup probe failed: Get "http://10.133.0.183:15021/healthz/ready": dial tcp 10.133.0.183:15021: connect: connection refused 15m Warning ProbeError pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Startup probe error: Get "http://10.133.0.183:15021/healthz/ready": dial tcp 10.133.0.183:15021: connect: connection refused... 15m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-ad23c7-777db74db7-v2896 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 Stopping container payload-processing 15m Normal Killing pod/payload-processing-e2e-ipp-a-7aa217-7bc848b57b-psqwk Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-a-7aa217-785d964d6c-rqhfz Stopping container payload-pre-processing 15m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Stopping container istio-proxy 15m Normal Created pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Container created 15m Normal Started pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Container started 15m Normal Pulled pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-qkmqx Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-jmntj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 15m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff Created pod: payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-jmntj 15m Normal Started pod/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-jmntj Container started 15m Normal Pulled pod/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-pmhnj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 15m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd Error creating: pods "payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-be6b74: serviceaccount "payload-processing-e2e-ipp-b-be6b74" not found 15m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd Created pod: payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-pmhnj 15m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-be6b74 Scaled up replica set payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd from 0 to 1 15m Normal Started pod/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-pmhnj Container started 15m Normal Created pod/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-jmntj Container created 15m Normal AddedInterface pod/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-pmhnj Add eth0 [10.132.1.37/23] from ovn-kubernetes 15m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff Error creating: pods "payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-be6b74: serviceaccount "payload-processing-e2e-ipp-b-be6b74" not found 15m Normal Created pod/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-pmhnj Container created 15m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-be6b74 Scaled up replica set payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff from 0 to 1 15m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-jmntj Add eth0 [10.134.1.26/23] from ovn-kubernetes 15m Warning ProbeError pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-ktwlr Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal Killing pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql Stopping container istio-proxy 15m Warning ProbeError pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Startup probe error: Get "http://10.132.1.23:15021/healthz/ready": dial tcp 10.132.1.23:15021: connect: connection refused... 15m Warning Unhealthy pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Startup probe failed: Get "http://10.132.1.23:15021/healthz/ready": dial tcp 10.132.1.23:15021: connect: connection refused 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c2b092-5757dbb7f9-zf2r5 Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-a-c2b092-54548c6cf4-2v2zd Stopping container payload-processing 15m Normal Killing pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-vxkk7 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-qxb79 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-jf58j Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-dr474 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-zfzrc Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Stopping container istio-proxy 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-be6b74-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-be6b74-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-8e6762-7745b85d8-bkft8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64 Readiness probe failed: timeout: failed to connect service "10.133.0.152:50052" within 1s: context deadline exceeded 15m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-7nfzk Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gj29r Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-gbw8d Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-hj9vq Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-nwqwp Stopping container istio-proxy 15m Warning Unhealthy pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Readiness probe failed: Get "http://10.132.0.244:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Warning ProbeError pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-58bj4 Readiness probe error: Get "http://10.132.0.244:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-aa49c5-69dfbdb5c9-vcb79 Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-b-aa49c5-c48688486-ljmsd Stopping container payload-processing 15m Warning ProbeError pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 Stopping container payload-processing 15m Warning Unhealthy pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-1b1531-596d8b977f-z9mj5 Stopping container payload-pre-processing 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-be6b74-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-c0a27e-6d7954cf9-tflx2 Stopping container payload-pre-processing 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-be6b74-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 15m Normal Killing pod/payload-processing-e2e-ipp-b-c0a27e-74bbd9584d-nxjsk Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-a-826518-5879559bc5-tf7wf Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-a-826518-7c7d6889c4-8gkdn Stopping container payload-processing 15m Normal Killing pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Stopping container istio-proxy 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-c0a27e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning ProbeError pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-ipp-b-aa49c5-openshift-default-6b56844555-6m8pc Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-c0a27e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-qz958 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-p2jpz Stopping container istio-proxy 15m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-frcfd Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-9gpf2 Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal Killing pod/e2e-ipp-a-826518-openshift-default-67bd854bff-5wppr Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km Stopping container istio-proxy 15m Warning Unhealthy pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-8tnvv Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-1b1531-7d7cf7c59b-f7r89 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk Stopping container payload-processing 14m Normal Killing pod/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h Stopping container payload-processing 14m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-worker-w4-27b5d5 Failed to update endpoint openshift-ingress/payload-processing-e2e-worker-w4-27b5d5: Operation cannot be fulfilled on endpoints "payload-processing-e2e-worker-w4-27b5d5": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-worker-w4-27b5d5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2cc36a82-9cf7-4e8e-8c3b-8605e1492dad, UID in object meta: 14m Normal Killing pod/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb Stopping container payload-pre-processing 14m Normal Killing pod/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-ipp-b-be6b74-5cdbb499ff-jmntj Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-ipp-b-be6b74-6dbc7d9ddd-pmhnj Stopping container payload-processing 14m Normal Killing pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-5rncs Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-77zds Stopping container istio-proxy 14m Normal Killing pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Stopping container istio-proxy 14m Warning Unhealthy pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning Unhealthy pod/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning ProbeError pod/e2e-ipp-b-be6b74-openshift-default-954dcdd97-w4h6k Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-shared-b-w2-8f3ff4 Failed to update endpoint openshift-ingress/payload-processing-e2e-shared-b-w2-8f3ff4: Operation cannot be fulfilled on endpoints "payload-processing-e2e-shared-b-w2-8f3ff4": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-shared-b-w2-8f3ff4, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a50f21a7-c238-4e87-ab21-ee4b1e992eaf, UID in object meta: 14m Normal Killing pod/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx Stopping container payload-pre-processing 14m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-b281d5-d8d4b4ff6-46rzk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-b281d5-6b596cff6c-gth9h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Deleted pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 14m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l Stopping container istio-proxy 14m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-fd7c77-openshift-default New size: 7; reason: All metrics below target 14m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Deleted pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-v7r7l 14m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-7dj59 Stopping container istio-proxy 14m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-fd7c77-openshift-default-6995885647 Deleted pod: e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 14m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-fd7c77-openshift-default Scaled down replica set e2e-shared-b-w3-fd7c77-openshift-default-6995885647 from 10 to 7 14m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-gz269 Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-shared-a-w3-ddb6ea Failed to update endpoint openshift-ingress/payload-processing-e2e-shared-a-w3-ddb6ea: Operation cannot be fulfilled on endpoints "payload-processing-e2e-shared-a-w3-ddb6ea": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-shared-a-w3-ddb6ea, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: fb0ac1a5-ac39-45f0-b3a7-246c4a65b24a, UID in object meta: 14m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp Stopping container payload-processing 14m Warning Unhealthy pod/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-hv7rr Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-l5mvb Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-d9gs9 Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-jdlkj Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w2-8f3ff4-openshift-default-74bf967ff9-9rf8n Stopping container istio-proxy 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-8f3ff4-849b4fdb59-q8nxx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-8f3ff4-89c77ff84-nszrp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-whjp6 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-qvxf5 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Stopping container istio-proxy 14m Warning ProbeError pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/e2e-shared-a-w3-ddb6ea-openshift-default-c489869-dclrd Readiness probe failed: HTTP probe failed with statuscode: 503 14m Normal Killing pod/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 Stopping container payload-processing 14m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-shared-b-w3-fd7c77 Failed to update endpoint openshift-ingress/payload-processing-e2e-shared-b-w3-fd7c77: Operation cannot be fulfilled on endpoints "payload-processing-e2e-shared-b-w3-fd7c77": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-shared-b-w3-fd7c77, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4f46920d-8729-4492-ae58-aa57435a741f, UID in object meta: 14m Normal Killing pod/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 Stopping container payload-pre-processing 14m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-ddb6ea-849d9cf6b-87fsp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-ddb6ea-75564d979-6j2pr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-vjnxg Stopping container istio-proxy 13m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Stopping container istio-proxy 13m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-5j29p Stopping container istio-proxy 13m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d Stopping container istio-proxy 13m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-xb7mw Stopping container istio-proxy 13m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Stopping container istio-proxy 13m Normal Killing pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-l4g58 Stopping container istio-proxy 13m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Readiness probe error: HTTP probe failed with statuscode: 503... 13m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-crtpl Readiness probe failed: HTTP probe failed with statuscode: 503 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-39b867-5b4c4b4bf9-kxww8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-processing-e2e-worker-w2-39b867-6bb6768bb4-9n2s6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Readiness probe failed: HTTP probe failed with statuscode: 503 13m Warning ProbeError pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-bwslp Readiness probe error: HTTP probe failed with statuscode: 503... 13m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf Stopping container istio-proxy 13m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4sh2l Stopping container istio-proxy 13m Normal Killing pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-ct8nr Stopping container istio-proxy 13m Normal Killing pod/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd Stopping container payload-pre-processing 13m Normal Killing pod/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 Stopping container payload-processing 13m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-fd7c77-5cdd779744-f5kzn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l Stopping container istio-proxy 13m Normal Killing pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Stopping container istio-proxy 13m Warning ProbeError pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Readiness probe error: HTTP probe failed with statuscode: 503... 13m Warning Unhealthy pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-l4486 Readiness probe failed: HTTP probe failed with statuscode: 503 12m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-855f9574cf from 6 to 10 12m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-xlxqw 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-6hpnk 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-vs4nx 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-855f9574cf Created pod: maas-default-gateway-openshift-default-855f9574cf-m5mbs 12m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Container created 12m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Container started 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Add eth0 [10.132.1.51/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Container created 12m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Container started 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Add eth0 [10.134.1.35/23] from ovn-kubernetes 12m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Container created 12m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Container created 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Add eth0 [10.134.1.36/23] from ovn-kubernetes 12m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Container started 12m Normal Started pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Container started 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Add eth0 [10.133.0.190/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Startup probe error: Get "http://10.134.1.35:15021/healthz/ready": dial tcp 10.134.1.35:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Startup probe failed: Get "http://10.134.1.35:15021/healthz/ready": dial tcp 10.134.1.35:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Startup probe failed: Get "http://10.134.1.36:15021/healthz/ready": dial tcp 10.134.1.36:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Startup probe error: Get "http://10.133.0.190:15021/healthz/ready": dial tcp 10.133.0.190:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Startup probe failed: Get "http://10.132.1.51:15021/healthz/ready": dial tcp 10.132.1.51:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Startup probe failed: Get "http://10.133.0.190:15021/healthz/ready": dial tcp 10.133.0.190:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Startup probe error: Get "http://10.134.1.36:15021/healthz/ready": dial tcp 10.134.1.36:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Startup probe error: Get "http://10.132.1.51:15021/healthz/ready": dial tcp 10.132.1.51:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Startup probe error: Get "http://10.133.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Startup probe error: Get "http://10.132.1.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-vs4nx Startup probe failed: Get "http://10.132.1.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk Startup probe failed: Get "http://10.133.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Startup probe failed: Get "http://10.134.1.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-m5mbs Startup probe error: Get "http://10.134.1.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Startup probe error: Get "http://10.134.1.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-xlxqw Startup probe failed: Get "http://10.134.1.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 11m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-b7qjj 11m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj Stopping container discovery 11m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 10m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-kk2r9 10m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 10m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 10m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-kk2r9 Stopping container discovery 8m52s Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-5j4cl Stopping container discovery 8m52s Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-5j4cl 8m52s Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 8m52s Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 8m41s Normal AddedLabel gateway/e2e-worker-single-main-a89c71 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-a89c71 8m41s Normal ScalingReplicaSet deployment/e2e-worker-single-main-a89c71-openshift-default Scaled up replica set e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9 from 0 to 1 8m41s Normal SuccessfulCreate replicaset/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9 Created pod: e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c 8m41s Normal AddedInterface pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Add eth0 [10.134.1.50/23] from ovn-kubernetes 8m37s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-a89c71 Scaled up replica set payload-processing-e2e-worker-single-main-a89c71-7456c5cfdb from 0 to 1 8m37s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-a89c71-6d898c6cdc Error creating: pods "payload-pre-processing-e2e-worker-single-main-a89c71-6d898c6cdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a89c71: serviceaccount "payload-processing-e2e-worker-single-main-a89c71" not found 8m37s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-a89c71 Scaled up replica set payload-pre-processing-e2e-worker-single-main-a89c71-6d898c6cdc from 0 to 1 8m37s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-a89c71-6d898c6cdc Created pod: payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c 8m37s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdb Created pod: payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz 8m37s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdb Error creating: pods "payload-processing-e2e-worker-single-main-a89c71-7456c5cfdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a89c71: serviceaccount "payload-processing-e2e-worker-single-main-a89c71" not found 8m36s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c Add eth0 [10.134.1.51/23] from ovn-kubernetes 8m36s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz Add eth0 [10.132.1.52/23] from ovn-kubernetes 8m36s Normal Pulled pod/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m36s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m36s Normal Created pod/payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c Container created 8m36s Normal Started pod/payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c Container started 8m36s Normal Created pod/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz Container created 8m36s Normal Started pod/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz Container started 8m25s Normal Pulled pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m25s Normal Created pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Container created 8m25s Normal Started pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Container started 8m20s Warning ProbeError pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Startup probe error: Get "http://10.134.1.50:15021/healthz/ready": dial tcp 10.134.1.50:15021: connect: connection refused... 8m20s Warning Unhealthy pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Startup probe failed: Get "http://10.134.1.50:15021/healthz/ready": dial tcp 10.134.1.50:15021: connect: connection refused 8m11s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-a89c71-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m11s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-a89c71-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m56s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-a89c71-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m56s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-a89c71-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m44s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c Stopping container payload-pre-processing 7m44s Normal Killing pod/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz Stopping container payload-processing 7m36s Normal Killing pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Stopping container istio-proxy 7m34s Warning Unhealthy pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Readiness probe failed: HTTP probe failed with statuscode: 503 7m34s Warning ProbeError pod/e2e-worker-single-main-a89c71-openshift-default-7f5ff58fb9d2r5c Readiness probe error: HTTP probe failed with statuscode: 503... 7m25s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-a89c71-7456c5cfdcrrgz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m24s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-a89c71-6d898kpd7c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m22s Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Readiness probe error: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m22s Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-hcvdj Readiness probe failed: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m37s Warning Unhealthy pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Readiness probe failed: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m37s Warning ProbeError pod/maas-default-gateway-openshift-default-855f9574cf-pthrg Readiness probe error: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m24s Normal ScalingReplicaSet deployment/e2e-worker-single-main-5310d3-openshift-default Scaled up replica set e2e-worker-single-main-5310d3-openshift-default-5668c5bd4f from 0 to 1 5m24s Normal AddedLabel gateway/e2e-worker-single-main-5310d3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-5310d3 5m24s Normal SuccessfulCreate replicaset/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4f Created pod: e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv 5m23s Normal AddedInterface pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Add eth0 [10.134.1.57/23] from ovn-kubernetes 5m19s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-5310d3-6775b77c9f Created pod: payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp 5m19s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-5310d3-5cd75c848 Created pod: payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp 5m19s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-5310d3 Scaled up replica set payload-processing-e2e-worker-single-main-5310d3-5cd75c848 from 0 to 1 5m19s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-5310d3-5cd75c848 Error creating: pods "payload-processing-e2e-worker-single-main-5310d3-5cd75c848-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-5310d3: serviceaccount "payload-processing-e2e-worker-single-main-5310d3" not found 5m19s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-5310d3-6775b77c9f Error creating: pods "payload-pre-processing-e2e-worker-single-main-5310d3-6775b77c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-5310d3: serviceaccount "payload-processing-e2e-worker-single-main-5310d3" not found 5m19s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-5310d3 Scaled up replica set payload-pre-processing-e2e-worker-single-main-5310d3-6775b77c9f from 0 to 1 5m18s Normal Pulled pod/payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m18s Normal Created pod/payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp Container created 5m18s Normal Created pod/payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp Container created 5m18s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp Add eth0 [10.134.1.58/23] from ovn-kubernetes 5m18s Normal Started pod/payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp Container started 5m18s Normal Started pod/payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp Container started 5m18s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m18s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp Add eth0 [10.132.1.54/23] from ovn-kubernetes 5m7s Warning Unhealthy pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Readiness probe failed: Get "http://10.134.1.57:15021/healthz/ready": dial tcp 10.134.1.57:15021: connect: connection refused 5m7s Normal Started pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Container started 5m7s Normal Pulled pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m7s Normal Created pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Container created 5m7s Warning ProbeError pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Readiness probe error: Get "http://10.134.1.57:15021/healthz/ready": dial tcp 10.134.1.57:15021: connect: connection refused... 5m1s Warning Unhealthy pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Startup probe failed: Get "http://10.134.1.57:15021/healthz/ready": dial tcp 10.134.1.57:15021: connect: connection refused 5m1s Warning ProbeError pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Startup probe error: Get "http://10.134.1.57:15021/healthz/ready": dial tcp 10.134.1.57:15021: connect: connection refused... 4m39s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-5310d3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m39s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-5310d3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m25s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp Stopping container payload-pre-processing 4m25s Normal Killing pod/payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp Stopping container payload-processing 4m17s Normal Killing pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Stopping container istio-proxy 4m15s Warning ProbeError pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Readiness probe error: HTTP probe failed with statuscode: 503... 4m15s Warning Unhealthy pod/e2e-worker-single-main-5310d3-openshift-default-5668c5bd4fb4xpv Readiness probe failed: HTTP probe failed with statuscode: 503 4m7s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-5310d3-6775bzk7zp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m6s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-5310d3-5cd75c848f8jtp Readiness probe failed: service unhealthy (responded with "NOT_SERVING")