=== events in openshift-ingress (2026-10-31T20:30:52Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 49m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn to ip-10-0-138-35.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh to ip-10-0-138-35.us-east-2.compute.internal 56m Normal Scheduled pod/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 Successfully assigned openshift-ingress/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 to ip-10-0-131-94.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr to ip-10-0-142-116.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd to ip-10-0-131-94.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Warning FailedScheduling pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Warning FailedScheduling pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 57m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h to ip-10-0-142-116.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv to ip-10-0-138-35.us-east-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr to ip-10-0-142-116.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks Successfully assigned openshift-ingress/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks to ip-10-0-131-94.us-east-2.compute.internal 44m Warning FailedScheduling pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz to ip-10-0-138-35.us-east-2.compute.internal 11m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg to ip-10-0-131-94.us-east-2.compute.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Successfully assigned openshift-ingress/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 to ip-10-0-131-94.us-east-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x to ip-10-0-138-35.us-east-2.compute.internal 8m15s Normal Scheduled pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Successfully assigned openshift-ingress/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj to ip-10-0-131-94.us-east-2.compute.internal 5m20s Normal Scheduled pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Successfully assigned openshift-ingress/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs to ip-10-0-131-94.us-east-2.compute.internal 45m Warning FailedScheduling pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r Successfully assigned openshift-ingress/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r to ip-10-0-138-35.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Warning FailedScheduling pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 62m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-tbx78 to ip-10-0-138-35.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Successfully assigned openshift-ingress/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 to ip-10-0-131-94.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 Successfully assigned openshift-ingress/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg to ip-10-0-131-94.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg Successfully assigned openshift-ingress/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg to ip-10-0-138-35.us-east-2.compute.internal 44m Warning FailedScheduling pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 52m Normal Scheduled pod/payload-processing-e2e-api-a-9e323f-778fbb59d7-49m62 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-9e323f-778fbb59d7-49m62 to ip-10-0-138-35.us-east-2.compute.internal 45m Warning FailedScheduling pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Warning FailedScheduling pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 63m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-t5m2j to ip-10-0-138-35.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-t27sn to ip-10-0-131-94.us-east-2.compute.internal 71m Normal Scheduled pod/router-default-8c75d8d7f-tzzdl Successfully assigned openshift-ingress/router-default-8c75d8d7f-tzzdl to ip-10-0-142-116.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-ait-63eab93e-openshift-default-6d4c965c9c-lwwjn Successfully assigned openshift-ingress/e2e-ait-63eab93e-openshift-default-6d4c965c9c-lwwjn to ip-10-0-131-94.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-mt-67adab47-998479859-c8zq5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-67adab47-998479859-c8zq5 to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Successfully assigned openshift-ingress/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd to ip-10-0-142-116.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs Successfully assigned openshift-ingress/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs to ip-10-0-142-116.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl Successfully assigned openshift-ingress/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Successfully assigned openshift-ingress/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd to ip-10-0-142-116.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-migrate-85be723d-545664d4ff-7jldh Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-85be723d-545664d4ff-7jldh to ip-10-0-138-35.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Successfully assigned openshift-ingress/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng to ip-10-0-138-35.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq to ip-10-0-138-35.us-east-2.compute.internal 56m Normal Scheduled pod/e2e-ait-f0303f0d-openshift-default-5b5588d58f-c5sdh Successfully assigned openshift-ingress/e2e-ait-f0303f0d-openshift-default-5b5588d58f-c5sdh to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w to ip-10-0-142-116.us-east-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 to ip-10-0-142-116.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-qvj7p Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-qvj7p to ip-10-0-131-94.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk Successfully assigned openshift-ingress/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk to ip-10-0-131-94.us-east-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Successfully assigned openshift-ingress/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 to ip-10-0-131-94.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Successfully assigned openshift-ingress/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 to ip-10-0-138-35.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Successfully assigned openshift-ingress/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 to ip-10-0-138-35.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Successfully assigned openshift-ingress/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl to ip-10-0-131-94.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Successfully assigned openshift-ingress/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm to ip-10-0-131-94.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Successfully assigned openshift-ingress/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd to ip-10-0-131-94.us-east-2.compute.internal 72m Warning FailedScheduling pod/router-default-8c75d8d7f-tzzdl no nodes available to schedule pods 78m Warning FailedScheduling pod/router-default-8c75d8d7f-tzzdl no nodes available to schedule pods 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-hs4dc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-hs4dc to ip-10-0-138-35.us-east-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-cf2lr Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-cf2lr to ip-10-0-138-35.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Successfully assigned openshift-ingress/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-7krhv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-7krhv to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 Successfully assigned openshift-ingress/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 to ip-10-0-142-116.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-b9svq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-b9svq to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf to ip-10-0-142-116.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk to ip-10-0-138-35.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 to ip-10-0-142-116.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 to ip-10-0-131-94.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 to ip-10-0-142-116.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc to ip-10-0-131-94.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l to ip-10-0-131-94.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Successfully assigned openshift-ingress/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl to ip-10-0-131-94.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 to ip-10-0-131-94.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr to ip-10-0-142-116.us-east-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 to ip-10-0-142-116.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj to ip-10-0-138-35.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds to ip-10-0-138-35.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk to ip-10-0-142-116.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm to ip-10-0-142-116.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw to ip-10-0-138-35.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 to ip-10-0-138-35.us-east-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Successfully assigned openshift-ingress/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln to ip-10-0-131-94.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-hmqdf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-hmqdf to ip-10-0-138-35.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk to ip-10-0-131-94.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-5685f Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-5685f to ip-10-0-138-35.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 Successfully assigned openshift-ingress/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq to ip-10-0-142-116.us-east-2.compute.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-ft6fm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-ft6fm to ip-10-0-138-35.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 Successfully assigned openshift-ingress/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 to ip-10-0-131-94.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Successfully assigned openshift-ingress/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz to ip-10-0-142-116.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-4j9n2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-4j9n2 to ip-10-0-138-35.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-ld47d Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-ld47d to ip-10-0-131-94.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd to ip-10-0-131-94.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p Successfully assigned openshift-ingress/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p to ip-10-0-142-116.us-east-2.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-2c4l7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-2c4l7 to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 to ip-10-0-138-35.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Successfully assigned openshift-ingress/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b to ip-10-0-138-35.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-85np2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-85np2 to ip-10-0-142-116.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 Successfully assigned openshift-ingress/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 to ip-10-0-131-94.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 Successfully assigned openshift-ingress/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 to ip-10-0-138-35.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 Successfully assigned openshift-ingress/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 to ip-10-0-142-116.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Successfully assigned openshift-ingress/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq to ip-10-0-142-116.us-east-2.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv to ip-10-0-138-35.us-east-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt to ip-10-0-131-94.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-6xd4b Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-6xd4b to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 to ip-10-0-138-35.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Successfully assigned openshift-ingress/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 to ip-10-0-138-35.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr Successfully assigned openshift-ingress/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv Successfully assigned openshift-ingress/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv to ip-10-0-138-35.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 Successfully assigned openshift-ingress/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm to ip-10-0-138-35.us-east-2.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v to ip-10-0-142-116.us-east-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t to ip-10-0-142-116.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h Successfully assigned openshift-ingress/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt to ip-10-0-142-116.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Successfully assigned openshift-ingress/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk to ip-10-0-142-116.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Successfully assigned openshift-ingress/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf to ip-10-0-142-116.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 to ip-10-0-131-94.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Successfully assigned openshift-ingress/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs to ip-10-0-142-116.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr Successfully assigned openshift-ingress/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr to ip-10-0-142-116.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd Successfully assigned openshift-ingress/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-api-a-2d319a-7dcc698947-479ft Successfully assigned openshift-ingress/payload-processing-e2e-api-a-2d319a-7dcc698947-479ft to ip-10-0-131-94.us-east-2.compute.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-8jbqq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-8jbqq to ip-10-0-131-94.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Successfully assigned openshift-ingress/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 to ip-10-0-142-116.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Warning FailedScheduling pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Successfully assigned openshift-ingress/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp Successfully assigned openshift-ingress/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp to ip-10-0-138-35.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-nxcxx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-nxcxx to ip-10-0-138-35.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Successfully assigned openshift-ingress/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g to ip-10-0-138-35.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-bs8jm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-bs8jm to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s to ip-10-0-142-116.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Successfully assigned openshift-ingress/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 to ip-10-0-138-35.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg to ip-10-0-142-116.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 Successfully assigned openshift-ingress/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Successfully assigned openshift-ingress/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs to ip-10-0-142-116.us-east-2.compute.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 to ip-10-0-138-35.us-east-2.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq to ip-10-0-138-35.us-east-2.compute.internal 11m Normal Scheduled pod/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw to ip-10-0-131-94.us-east-2.compute.internal 8m9s Normal Scheduled pod/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x to ip-10-0-138-35.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Successfully assigned openshift-ingress/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 to ip-10-0-138-35.us-east-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn to ip-10-0-131-94.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 to ip-10-0-138-35.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Successfully assigned openshift-ingress/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t to ip-10-0-142-116.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb to ip-10-0-138-35.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Successfully assigned openshift-ingress/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Successfully assigned openshift-ingress/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 to ip-10-0-131-94.us-east-2.compute.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk to ip-10-0-138-35.us-east-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp to ip-10-0-131-94.us-east-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-h7nqs Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-h7nqs to ip-10-0-138-35.us-east-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 to ip-10-0-138-35.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Successfully assigned openshift-ingress/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 to ip-10-0-131-94.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Successfully assigned openshift-ingress/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj to ip-10-0-142-116.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 Successfully assigned openshift-ingress/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 to ip-10-0-142-116.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np Successfully assigned openshift-ingress/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np to ip-10-0-131-94.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 Successfully assigned openshift-ingress/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 to ip-10-0-131-94.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 Successfully assigned openshift-ingress/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 to ip-10-0-131-94.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Successfully assigned openshift-ingress/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc to ip-10-0-142-116.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Successfully assigned openshift-ingress/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-zwgd7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-zwgd7 to ip-10-0-131-94.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 to ip-10-0-138-35.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 Successfully assigned openshift-ingress/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 to ip-10-0-142-116.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-pgf8r Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-pgf8r to ip-10-0-131-94.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf to ip-10-0-131-94.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs to ip-10-0-142-116.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Successfully assigned openshift-ingress/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv to ip-10-0-142-116.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 Successfully assigned openshift-ingress/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 to ip-10-0-138-35.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl to ip-10-0-142-116.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm Successfully assigned openshift-ingress/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm to ip-10-0-131-94.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Successfully assigned openshift-ingress/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 to ip-10-0-131-94.us-east-2.compute.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt Successfully assigned openshift-ingress/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt to ip-10-0-142-116.us-east-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc to ip-10-0-142-116.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Successfully assigned openshift-ingress/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-mzfnm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-mzfnm to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Successfully assigned openshift-ingress/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 to ip-10-0-131-94.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Successfully assigned openshift-ingress/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 to ip-10-0-142-116.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw to ip-10-0-142-116.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj to ip-10-0-138-35.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg to ip-10-0-138-35.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt to ip-10-0-131-94.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf to ip-10-0-142-116.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm Successfully assigned openshift-ingress/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm to ip-10-0-131-94.us-east-2.compute.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj to ip-10-0-138-35.us-east-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 to ip-10-0-138-35.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg to ip-10-0-142-116.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 to ip-10-0-138-35.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 to ip-10-0-142-116.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-v5wmk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-v5wmk to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l to ip-10-0-142-116.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 to ip-10-0-131-94.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg to ip-10-0-131-94.us-east-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg to ip-10-0-138-35.us-east-2.compute.internal 8m9s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Successfully assigned openshift-ingress/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz to ip-10-0-131-94.us-east-2.compute.internal 56m Normal Scheduled pod/e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5-2lbp5 Successfully assigned openshift-ingress/e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5-2lbp5 to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-z4pwr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-z4pwr to ip-10-0-131-94.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Successfully assigned openshift-ingress/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt to ip-10-0-142-116.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-hfrgd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-hfrgd to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 to ip-10-0-142-116.us-east-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 to ip-10-0-142-116.us-east-2.compute.internal 66m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-84d4d45d7cmgld8 Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-84d4d45d7cmgld8 to ip-10-0-142-116.us-east-2.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Successfully assigned openshift-ingress/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf to ip-10-0-142-116.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z to ip-10-0-142-116.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Successfully assigned openshift-ingress/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 to ip-10-0-138-35.us-east-2.compute.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt to ip-10-0-142-116.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc to ip-10-0-131-94.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Successfully assigned openshift-ingress/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd to ip-10-0-131-94.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr to ip-10-0-131-94.us-east-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Successfully assigned openshift-ingress/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v to ip-10-0-131-94.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm Successfully assigned openshift-ingress/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm to ip-10-0-138-35.us-east-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 to ip-10-0-142-116.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Successfully assigned openshift-ingress/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n to ip-10-0-131-94.us-east-2.compute.internal 44m Warning FailedScheduling pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 31m Normal Scheduled pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv Successfully assigned openshift-ingress/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv to ip-10-0-138-35.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l Successfully assigned openshift-ingress/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l to ip-10-0-142-116.us-east-2.compute.internal 43m Warning FailedScheduling pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 50m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 to ip-10-0-131-94.us-east-2.compute.internal 67m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-mnrh7 to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz to ip-10-0-138-35.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-hq6bn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-hq6bn to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh to ip-10-0-142-116.us-east-2.compute.internal 5m16s Normal Scheduled pod/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-x8n6g Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-x8n6g to ip-10-0-131-94.us-east-2.compute.internal 62m Normal Scheduled pod/payload-pre-processing-7b98bb678c-htgzs Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-htgzs to ip-10-0-138-35.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Successfully assigned openshift-ingress/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 to ip-10-0-131-94.us-east-2.compute.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-5pq8p Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-5pq8p to ip-10-0-138-35.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Successfully assigned openshift-ingress/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 to ip-10-0-131-94.us-east-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-mt-db61066b-5cbb466f49-5zjw7 Successfully assigned openshift-ingress/payload-processing-e2e-mt-db61066b-5cbb466f49-5zjw7 to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Successfully assigned openshift-ingress/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h to ip-10-0-142-116.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt to ip-10-0-138-35.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Successfully assigned openshift-ingress/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb to ip-10-0-142-116.us-east-2.compute.internal 58m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-2q74f Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2q74f to ip-10-0-131-94.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Successfully assigned openshift-ingress/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc to ip-10-0-131-94.us-east-2.compute.internal 55m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bhgqx to ip-10-0-142-116.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Successfully assigned openshift-ingress/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j to ip-10-0-142-116.us-east-2.compute.internal 65m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-gzhhr Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gzhhr to ip-10-0-138-35.us-east-2.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz to ip-10-0-142-116.us-east-2.compute.internal 67m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-5cf4665f7f-bxq79 Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-5cf4665f7f-bxq79 to ip-10-0-142-116.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z to ip-10-0-138-35.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-qqmwf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-qqmwf to ip-10-0-142-116.us-east-2.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt to ip-10-0-138-35.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Successfully assigned openshift-ingress/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs to ip-10-0-131-94.us-east-2.compute.internal 51m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-l4p8z Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l4p8z to ip-10-0-142-116.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Successfully assigned openshift-ingress/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc to ip-10-0-138-35.us-east-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Successfully assigned openshift-ingress/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx to ip-10-0-142-116.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Successfully assigned openshift-ingress/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs Successfully assigned openshift-ingress/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs to ip-10-0-138-35.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Successfully assigned openshift-ingress/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx to ip-10-0-142-116.us-east-2.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw to ip-10-0-138-35.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-wntjb Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-wntjb to ip-10-0-138-35.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Successfully assigned openshift-ingress/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 to ip-10-0-142-116.us-east-2.compute.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr to ip-10-0-138-35.us-east-2.compute.internal 66m Normal Scheduled pod/kube-auth-proxy-849d94dfbb-ff72l Successfully assigned openshift-ingress/kube-auth-proxy-849d94dfbb-ff72l to ip-10-0-138-35.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl Successfully assigned openshift-ingress/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl to ip-10-0-138-35.us-east-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn to ip-10-0-131-94.us-east-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-api-a-2d319a-d95b54579-hzwkl Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-2d319a-d95b54579-hzwkl to ip-10-0-131-94.us-east-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-kjwgx Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-kjwgx to ip-10-0-138-35.us-east-2.compute.internal 66m Normal Scheduled pod/kube-auth-proxy-849d94dfbb-l28r8 Successfully assigned openshift-ingress/kube-auth-proxy-849d94dfbb-l28r8 to ip-10-0-131-94.us-east-2.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 to ip-10-0-138-35.us-east-2.compute.internal 53m Normal Scheduled pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 Successfully assigned openshift-ingress/payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 to ip-10-0-138-35.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Successfully assigned openshift-ingress/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq to ip-10-0-131-94.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk Successfully assigned openshift-ingress/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk to ip-10-0-142-116.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-api-a-e73256-b9d6b9887-62lj9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-e73256-b9d6b9887-62lj9 to ip-10-0-138-35.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-c79ts Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-c79ts to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Successfully assigned openshift-ingress/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr to ip-10-0-142-116.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Successfully assigned openshift-ingress/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp to ip-10-0-131-94.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw to ip-10-0-131-94.us-east-2.compute.internal 62m Normal Scheduled pod/payload-processing-55fbbbf9dc-76hjw Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-76hjw to ip-10-0-131-94.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx Successfully assigned openshift-ingress/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx to ip-10-0-142-116.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj to ip-10-0-138-35.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-n2bkz Successfully assigned openshift-ingress/payload-processing-e2e-mt-67adab47-7ddbcc466c-n2bkz to ip-10-0-131-94.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-55znv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-55znv to ip-10-0-131-94.us-east-2.compute.internal 4m43s Normal Scheduled pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Successfully assigned openshift-ingress/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww to ip-10-0-131-94.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-68qbd to ip-10-0-131-94.us-east-2.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-6k6cd to ip-10-0-131-94.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk Successfully assigned openshift-ingress/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk to ip-10-0-142-116.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Successfully assigned openshift-ingress/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw to ip-10-0-142-116.us-east-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-894jp Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-894jp to ip-10-0-142-116.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z to ip-10-0-138-35.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc to ip-10-0-131-94.us-east-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 to ip-10-0-138-35.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-migrate-85be723d-578cfdbb9c-bh7qf Successfully assigned openshift-ingress/payload-processing-e2e-migrate-85be723d-578cfdbb9c-bh7qf to ip-10-0-138-35.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-d7647c-7457796869-bbpbw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-d7647c-7457796869-bbpbw to ip-10-0-138-35.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-d7647c-66bbb74874-4dsdn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-d7647c-66bbb74874-4dsdn to ip-10-0-131-94.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-mqgll Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-mqgll to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-lrnmf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-lrnmf to ip-10-0-131-94.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-a84fcb-bdbd55747-zkl6p Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-a84fcb-bdbd55747-zkl6p to ip-10-0-138-35.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt Successfully assigned openshift-ingress/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt to ip-10-0-131-94.us-east-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd to ip-10-0-138-35.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb Successfully assigned openshift-ingress/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb to ip-10-0-138-35.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-6ncmf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-6ncmf to ip-10-0-138-35.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Successfully assigned openshift-ingress/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 to ip-10-0-138-35.us-east-2.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-6qm4f to ip-10-0-142-116.us-east-2.compute.internal 43m Warning FailedScheduling pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs to ip-10-0-138-35.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Successfully assigned openshift-ingress/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb to ip-10-0-131-94.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg to ip-10-0-131-94.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l to ip-10-0-138-35.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv to ip-10-0-131-94.us-east-2.compute.internal 45m Warning FailedScheduling pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Warning FailedScheduling pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Normal Scheduled pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf Successfully assigned openshift-ingress/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf to ip-10-0-138-35.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-v2fc5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-v2fc5 to ip-10-0-131-94.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-dqxhs Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-dqxhs to ip-10-0-138-35.us-east-2.compute.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-7fe845-6db886499c-vfjqn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-7fe845-6db886499c-vfjqn to ip-10-0-138-35.us-east-2.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-ipp-b-5f3eff-7655b4885c-k6dn9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-5f3eff-7655b4885c-k6dn9 to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-mc8nv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-mc8nv to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg to ip-10-0-131-94.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-b-01afb5-7956f95fdf-4sgpr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-01afb5-7956f95fdf-4sgpr to ip-10-0-131-94.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Successfully assigned openshift-ingress/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 to ip-10-0-142-116.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w Successfully assigned openshift-ingress/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w to ip-10-0-131-94.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-f28d48-7745df5fc-xzcb6 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f28d48-7745df5fc-xzcb6 to ip-10-0-138-35.us-east-2.compute.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-f28d48-6fc75df778-4x22w Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f28d48-6fc75df778-4x22w to ip-10-0-131-94.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-xdjpr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-xdjpr to ip-10-0-131-94.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-c173d8-6d47cf995d-2bjft Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c173d8-6d47cf995d-2bjft to ip-10-0-131-94.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-qnnw8 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-qnnw8 to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-2bqtm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-2bqtm to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 to ip-10-0-131-94.us-east-2.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-ipp-a-26cd59-84899b8c48-bgz8c Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-26cd59-84899b8c48-bgz8c to ip-10-0-131-94.us-east-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx to ip-10-0-131-94.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-j7z8j Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-j7z8j to ip-10-0-131-94.us-east-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj to ip-10-0-138-35.us-east-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-ipp-a-1f2781-76cd6d497f-sbqjg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-1f2781-76cd6d497f-sbqjg to ip-10-0-142-116.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm to ip-10-0-131-94.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-d8c398-546ff884f-pr754 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-d8c398-546ff884f-pr754 to ip-10-0-138-35.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-d45f6c-59d8bb7487-tcsb7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-d45f6c-59d8bb7487-tcsb7 to ip-10-0-138-35.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-6rwmb to ip-10-0-131-94.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Successfully assigned openshift-ingress/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb to ip-10-0-142-116.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc to ip-10-0-138-35.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-b-8dcfa8-86d859f49f-j5f54 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-8dcfa8-86d859f49f-j5f54 to ip-10-0-131-94.us-east-2.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-9th7f to ip-10-0-131-94.us-east-2.compute.internal 42m Warning FailedScheduling pod/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll to ip-10-0-131-94.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Successfully assigned openshift-ingress/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq to ip-10-0-142-116.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j to ip-10-0-142-116.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Successfully assigned openshift-ingress/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p to ip-10-0-131-94.us-east-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-b-7c892a-58845567f5-kgksv Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-7c892a-58845567f5-kgksv to ip-10-0-142-116.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-x9g5w Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-x9g5w to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-inf-a-a4e312-7b78fb8665-b9pn6 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-a4e312-7b78fb8665-b9pn6 to ip-10-0-131-94.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 to ip-10-0-131-94.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 to ip-10-0-138-35.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn to ip-10-0-131-94.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch Successfully assigned openshift-ingress/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch to ip-10-0-142-116.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 to ip-10-0-131-94.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc to ip-10-0-131-94.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb to ip-10-0-138-35.us-east-2.compute.internal 45m Warning FailedScheduling pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 63m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-dh8nw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-dh8nw to ip-10-0-138-35.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-rj87s Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-rj87s to ip-10-0-131-94.us-east-2.compute.internal 4m38s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-6jzkt Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-6jzkt to ip-10-0-131-94.us-east-2.compute.internal 45m Warning FailedScheduling pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Warning FailedScheduling pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Normal Scheduled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v Successfully assigned openshift-ingress/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v to ip-10-0-131-94.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l Successfully assigned openshift-ingress/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l to ip-10-0-138-35.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v to ip-10-0-138-35.us-east-2.compute.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-wp72k Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-wp72k to ip-10-0-138-35.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Successfully assigned openshift-ingress/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt to ip-10-0-131-94.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk to ip-10-0-138-35.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Successfully assigned openshift-ingress/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z to ip-10-0-131-94.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f92ld to ip-10-0-142-116.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v Successfully assigned openshift-ingress/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v to ip-10-0-138-35.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-fhccm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fhccm to ip-10-0-142-116.us-east-2.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-gj6s5 to ip-10-0-142-116.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 Successfully assigned openshift-ingress/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 to ip-10-0-131-94.us-east-2.compute.internal 55m Normal Scheduled pod/payload-processing-e2e-derive-0b9ff685-b95f54655-qk54m Successfully assigned openshift-ingress/payload-processing-e2e-derive-0b9ff685-b95f54655-qk54m to ip-10-0-138-35.us-east-2.compute.internal 4m38s Normal Scheduled pod/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-9hndx Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-9hndx to ip-10-0-138-35.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Normal Scheduled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Successfully assigned openshift-ingress/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc to ip-10-0-142-116.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n Successfully assigned openshift-ingress/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n to ip-10-0-131-94.us-east-2.compute.internal 43m Warning FailedScheduling pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 56m Normal Scheduled pod/payload-processing-e2e-ait-1cc6560c-7f7db84869-wln55 Successfully assigned openshift-ingress/payload-processing-e2e-ait-1cc6560c-7f7db84869-wln55 to ip-10-0-138-35.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf Successfully assigned openshift-ingress/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf to ip-10-0-142-116.us-east-2.compute.internal 5m16s Normal Scheduled pod/payload-processing-e2e-ait-4554a17c-bfdd8df5-89k9l Successfully assigned openshift-ingress/payload-processing-e2e-ait-4554a17c-bfdd8df5-89k9l to ip-10-0-138-35.us-east-2.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-l6wqz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-l6wqz to ip-10-0-131-94.us-east-2.compute.internal 55m Normal Scheduled pod/payload-processing-e2e-ait-63eab93e-5bdf4b59f9-bsxqw Successfully assigned openshift-ingress/payload-processing-e2e-ait-63eab93e-5bdf4b59f9-bsxqw to ip-10-0-138-35.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-api-b-d4b21c-764698f6f-dt2q6 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-d4b21c-764698f6f-dt2q6 to ip-10-0-131-94.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj Successfully assigned openshift-ingress/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj to ip-10-0-142-116.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-api-b-a813e2-dbc7795cc-vqxqm Successfully assigned openshift-ingress/payload-processing-e2e-api-b-a813e2-dbc7795cc-vqxqm to ip-10-0-131-94.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Successfully assigned openshift-ingress/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq to ip-10-0-142-116.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g Successfully assigned openshift-ingress/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g to ip-10-0-131-94.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 43m Warning FailedScheduling pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 67m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-w7w5h to ip-10-0-142-116.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-lwvpn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwvpn to ip-10-0-131-94.us-east-2.compute.internal 45m Warning FailedScheduling pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Normal Scheduled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Successfully assigned openshift-ingress/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs to ip-10-0-138-35.us-east-2.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-msrfk to ip-10-0-131-94.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-api-b-1c0c1d-758fdd696f-ff9zq Successfully assigned openshift-ingress/payload-processing-e2e-api-b-1c0c1d-758fdd696f-ff9zq to ip-10-0-142-116.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-vkszn to ip-10-0-131-94.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-vj5qn to ip-10-0-131-94.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-ait-f0303f0d-6f65d9f4c-z7c4z Successfully assigned openshift-ingress/payload-processing-e2e-ait-f0303f0d-6f65d9f4c-z7c4z to ip-10-0-138-35.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb to ip-10-0-142-116.us-east-2.compute.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 to ip-10-0-138-35.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Successfully assigned openshift-ingress/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b to ip-10-0-131-94.us-east-2.compute.internal 45m Warning FailedScheduling pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 50m Normal Scheduled pod/payload-processing-e2e-api-a-e73256-57b5c8849f-n9qzl Successfully assigned openshift-ingress/payload-processing-e2e-api-a-e73256-57b5c8849f-n9qzl to ip-10-0-131-94.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Successfully assigned openshift-ingress/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 to ip-10-0-131-94.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-pdvwj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-pdvwj to ip-10-0-138-35.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-qgfcs to ip-10-0-142-116.us-east-2.compute.internal 78m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-8c75d8d7f from 0 to 1 78m Warning FailedCreate replicaset/router-default-8c75d8d7f Error creating: pods "router-default-8c75d8d7f-" is forbidden: error fetching namespace "openshift-ingress": unable to find annotation openshift.io/sa.scc.uid-range 78m Normal SuccessfulCreate replicaset/router-default-8c75d8d7f Created pod: router-default-8c75d8d7f-tzzdl 78m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 78m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 77m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 71m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 71m Warning FailedMount pod/router-default-8c75d8d7f-tzzdl MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 71m Warning FailedMount pod/router-default-8c75d8d7f-tzzdl MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 71m Warning FailedMount pod/router-default-8c75d8d7f-tzzdl MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 71m Warning FailedMount pod/router-default-8c75d8d7f-tzzdl MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 71m Normal Created pod/router-default-8c75d8d7f-tzzdl Container created 71m Normal Started pod/router-default-8c75d8d7f-tzzdl Container started 71m Normal Pulled pod/router-default-8c75d8d7f-tzzdl Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 71m Normal AddedInterface pod/router-default-8c75d8d7f-tzzdl Add eth0 [10.132.0.13/23] from ovn-kubernetes 67m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 67m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 67m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-mnrh7 67m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Add eth0 [10.133.0.21/23] from ovn-kubernetes 67m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 67m Normal Created pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Container created 67m Normal Started pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Container started 67m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.447s (3.447s including waiting). Image size: 185476477 bytes. 67m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-5cf4665f7f Created pod: openshift-ai-inference-openshift-ai-inference-5cf4665f7f-bxq79 67m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 67m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Readiness probe error: HTTP probe failed with statuscode: 503... 67m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Readiness probe failed: HTTP probe failed with statuscode: 503 67m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-w7w5h 67m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75d7c785f from 0 to 1 67m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-5cf4665f7f from 0 to 1 67m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-5cf4665f7f-bxq79 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 67m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Add eth0 [10.132.0.31/23] from ovn-kubernetes 67m Normal Pulling pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 67m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-5cf4665f7f-bxq79 Add eth0 [10.132.0.33/23] from ovn-kubernetes 67m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 67m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.624s (8.624s including waiting). Image size: 183555333 bytes. 67m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-5cf4665f7f-bxq79 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.52s (8.52s including waiting). Image size: 183555333 bytes. 67m Normal Started pod/openshift-ai-inference-openshift-ai-inference-5cf4665f7f-bxq79 Container started 67m Normal Created pod/openshift-ai-inference-openshift-ai-inference-5cf4665f7f-bxq79 Container created 67m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-849d94dfbb from 0 to 2 66m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-84d4d45d7c Created pod: data-science-gateway-data-science-gateway-class-84d4d45d7cmgld8 66m Normal SuccessfulCreate replicaset/kube-auth-proxy-849d94dfbb Created pod: kube-auth-proxy-849d94dfbb-ff72l 66m Normal SuccessfulCreate replicaset/kube-auth-proxy-849d94dfbb Created pod: kube-auth-proxy-849d94dfbb-l28r8 66m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-84d4d45d7c from 0 to 1 66m Normal Pulling pod/kube-auth-proxy-849d94dfbb-l28r8 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 66m Normal Pulling pod/kube-auth-proxy-849d94dfbb-ff72l Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 66m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-84d4d45d7cmgld8 Add eth0 [10.132.0.40/23] from ovn-kubernetes 66m Normal Pulled pod/data-science-gateway-data-science-gateway-class-84d4d45d7cmgld8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/data-science-gateway-data-science-gateway-class-84d4d45d7cmgld8 Container created 66m Normal Started pod/data-science-gateway-data-science-gateway-class-84d4d45d7cmgld8 Container started 66m Normal AddedInterface pod/kube-auth-proxy-849d94dfbb-l28r8 Add eth0 [10.133.0.23/23] from ovn-kubernetes 66m Normal AddedInterface pod/kube-auth-proxy-849d94dfbb-ff72l Add eth0 [10.134.0.27/23] from ovn-kubernetes 66m Normal Started pod/kube-auth-proxy-849d94dfbb-l28r8 Container started 66m Normal Pulled pod/kube-auth-proxy-849d94dfbb-ff72l Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.494s (2.494s including waiting). Image size: 188904677 bytes. 66m Normal Created pod/kube-auth-proxy-849d94dfbb-ff72l Container created 66m Normal Started pod/kube-auth-proxy-849d94dfbb-ff72l Container started 66m Normal Pulled pod/kube-auth-proxy-849d94dfbb-l28r8 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.318s (2.318s including waiting). Image size: 188904677 bytes. 66m Normal Created pod/kube-auth-proxy-849d94dfbb-l28r8 Container created 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-gzhhr Add eth0 [10.134.0.36/23] from ovn-kubernetes 65m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-gzhhr Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 65m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-gzhhr 65m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 65m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-gzhhr Container started 65m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-gzhhr Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.639s (4.639s including waiting). Image size: 185476477 bytes. 65m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-gzhhr Container created 65m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 65m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-mnrh7 65m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-mnrh7 Stopping container discovery 65m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Container started 65m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Container created 65m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Startup probe error: Get "http://10.132.0.31:15021/healthz/ready": dial tcp 10.132.0.31:15021: connect: connection refused... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Startup probe failed: Get "http://10.132.0.31:15021/healthz/ready": dial tcp 10.132.0.31:15021: connect: connection refused 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-t5m2j 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-msrfk 63m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75d7c785f from 1 to 3 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Add eth0 [10.134.0.38/23] from ovn-kubernetes 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Add eth0 [10.133.0.38/23] from ovn-kubernetes 63m Normal Pulling pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 63m Normal Pulling pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 63m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.206s (3.206s including waiting). Image size: 183555333 bytes. 63m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Container created 63m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.273s (3.273s including waiting). Image size: 183555333 bytes. 63m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Container started 63m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Container created 63m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Container started 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 63m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused... 63m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-gj6s5 63m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-dh8nw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-dh8nw Container created 63m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-l6wqz Container created 63m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-l6wqz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-l6wqz Add eth0 [10.133.0.39/23] from ovn-kubernetes 63m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-l6wqz Container started 63m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-dh8nw Container started 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Add eth0 [10.132.0.46/23] from ovn-kubernetes 63m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Container started 63m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Container created 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-dh8nw Add eth0 [10.134.0.40/23] from ovn-kubernetes 63m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-l6wqz 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-dh8nw 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-dh8nw Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-l6wqz Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-l6wqz Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused... 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-dh8nw Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 62m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 62m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Container created 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Add eth0 [10.133.0.42/23] from ovn-kubernetes 62m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Container created 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Add eth0 [10.133.0.43/23] from ovn-kubernetes 62m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Add eth0 [10.134.0.41/23] from ovn-kubernetes 62m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Container started 62m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Container created 62m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Container started 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Add eth0 [10.132.0.47/23] from ovn-kubernetes 62m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Container started 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-9th7f 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-6qm4f 62m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Container started 62m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Container created 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f (combined from similar events): Created pod: maas-default-gateway-openshift-default-75d7c785f-6k6cd 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-tbx78 62m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 62m Normal AddedInterface pod/payload-processing-55fbbbf9dc-76hjw Add eth0 [10.133.0.44/23] from ovn-kubernetes 62m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 62m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-htgzs Add eth0 [10.134.0.42/23] from ovn-kubernetes 62m Normal Pulling pod/payload-pre-processing-7b98bb678c-htgzs Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 62m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 62m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-76hjw 62m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-htgzs 62m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 62m Normal Pulling pod/payload-processing-55fbbbf9dc-76hjw Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused... 62m Normal Pulled pod/payload-processing-55fbbbf9dc-76hjw Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.458s (2.458s including waiting). Image size: 125353963 bytes. 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 62m Normal Started pod/payload-processing-55fbbbf9dc-76hjw Container started 62m Normal Pulled pod/payload-pre-processing-7b98bb678c-htgzs Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.264s (2.264s including waiting). Image size: 125353963 bytes. 62m Normal Created pod/payload-pre-processing-7b98bb678c-htgzs Container created 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 62m Normal Created pod/payload-processing-55fbbbf9dc-76hjw Container created 62m Normal Started pod/payload-pre-processing-7b98bb678c-htgzs Container started 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled up replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 0 to 1 61m Normal AddedLabel gateway/e2e-worker-w4-996c32 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-996c32 61m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled up replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 0 to 1 61m Normal AddedLabel gateway/e2e-worker-w0-23843f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-23843f 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled up replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 0 to 1 61m Normal AddedLabel gateway/e2e-worker-w1-8b083a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-8b083a 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln 61m Normal AddedLabel gateway/e2e-worker-w2-aeb292 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-aeb292 61m Normal AddedLabel gateway/e2e-worker-w6-8730d5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-8730d5 61m Normal AddedLabel gateway/e2e-worker-w3-23814d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-23814d 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled up replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 0 to 1 61m Normal AddedLabel gateway/e2e-worker-w5-5bbf75 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-5bbf75 61m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled up replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 0 to 1 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr 61m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Add eth0 [10.133.0.47/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm 61m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Add eth0 [10.132.0.49/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled up replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 0 to 1 61m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled up replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 0 to 1 61m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Add eth0 [10.132.0.50/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Add eth0 [10.133.0.45/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Add eth0 [10.133.0.46/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Add eth0 [10.132.0.48/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Add eth0 [10.133.0.48/23] from ovn-kubernetes 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9 Error creating: pods "payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-5bbf75: serviceaccount "payload-processing-e2e-worker-w5-5bbf75" not found 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9 Created pod: payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-5bbf75 Scaled up replica set payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9 from 0 to 1 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7 Error creating: pods "payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-5bbf75: serviceaccount "payload-processing-e2e-worker-w5-5bbf75" not found 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-5bbf75 Scaled up replica set payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7 from 0 to 1 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7 Created pod: payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5 Error creating: pods "payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-8b083a: serviceaccount "payload-processing-e2e-worker-w1-8b083a" not found 61m Normal Started pod/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 Container started 61m Normal Created pod/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 Container created 61m Normal Started pod/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw Container started 61m Normal Pulled pod/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5 Error creating: pods "payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-8b083a: serviceaccount "payload-processing-e2e-worker-w1-8b083a" not found 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5 Created pod: payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz 61m Normal Created pod/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw Container created 61m Normal AddedInterface pod/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 Add eth0 [10.134.0.45/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-8b083a Scaled up replica set payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5 from 0 to 1 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw Add eth0 [10.134.0.44/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5 Created pod: payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-8b083a Scaled up replica set payload-pre-processing-e2e-worker-w1-8b083a-76688799f5 from 0 to 1 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-996c32 Scaled up replica set payload-pre-processing-e2e-worker-w4-996c32-7585565cff from 0 to 1 61m Normal Started pod/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz Container started 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-996c32-5b66846755 Created pod: payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv 61m Normal AddedInterface pod/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g Add eth0 [10.134.0.47/23] from ovn-kubernetes 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-996c32-5b66846755 Error creating: pods "payload-processing-e2e-worker-w4-996c32-5b66846755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-996c32: serviceaccount "payload-processing-e2e-worker-w4-996c32" not found 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-996c32-7585565cff Error creating: pods "payload-pre-processing-e2e-worker-w4-996c32-7585565cff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-996c32: serviceaccount "payload-processing-e2e-worker-w4-996c32" not found 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz Add eth0 [10.134.0.48/23] from ovn-kubernetes 61m Normal Started pod/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g Container started 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-996c32-7585565cff Created pod: payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Created pod/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g Container created 61m Normal Pulled pod/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Created pod/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz Container created 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-996c32 Scaled up replica set payload-processing-e2e-worker-w4-996c32-5b66846755 from 0 to 1 61m Normal Started pod/payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv Container started 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5 Error creating: pods "payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-23843f: serviceaccount "payload-processing-e2e-worker-w0-23843f" not found 61m Normal Created pod/payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv Container created 61m Normal Pulled pod/payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv Add eth0 [10.134.0.49/23] from ovn-kubernetes 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt Add eth0 [10.134.0.50/23] from ovn-kubernetes 61m Normal Started pod/payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt Container started 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-23843f Scaled up replica set payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5 from 0 to 1 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5 Created pod: payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz 61m Normal Created pod/payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt Container created 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-23843f Scaled up replica set payload-processing-e2e-worker-w0-23843f-66cb64b99f from 0 to 1 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-23843f-66cb64b99f Created pod: payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-23843f-66cb64b99f Error creating: pods "payload-processing-e2e-worker-w0-23843f-66cb64b99f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-23843f: serviceaccount "payload-processing-e2e-worker-w0-23843f" not found 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf Created pod: payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf Error creating: pods "payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-8730d5: serviceaccount "payload-processing-e2e-worker-w6-8730d5" not found 61m Normal Pulled pod/payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq Add eth0 [10.134.0.51/23] from ovn-kubernetes 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-aeb292-7b88bf8765 Error creating: pods "payload-processing-e2e-worker-w2-aeb292-7b88bf8765-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-aeb292: serviceaccount "payload-processing-e2e-worker-w2-aeb292" not found 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c Error creating: pods "payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-8730d5: serviceaccount "payload-processing-e2e-worker-w6-8730d5" not found 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c Created pod: payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9 Error creating: pods "payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-aeb292: serviceaccount "payload-processing-e2e-worker-w2-aeb292" not found 61m Normal Started pod/payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq Container started 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9 Created pod: payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt 61m Normal Started pod/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz Container started 61m Normal Created pod/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz Container created 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz Add eth0 [10.134.0.52/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-aeb292 Scaled up replica set payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9 from 0 to 1 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-8730d5 Scaled up replica set payload-processing-e2e-worker-w6-8730d5-7f459f7c9c from 0 to 1 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-aeb292 Scaled up replica set payload-processing-e2e-worker-w2-aeb292-7b88bf8765 from 0 to 1 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-aeb292-7b88bf8765 Created pod: payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-8730d5 Scaled up replica set payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf from 0 to 1 61m Normal Created pod/payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq Container created 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Created pod/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 Container created 61m Normal Started pod/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 Container started 61m Normal AddedInterface pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Add eth0 [10.132.0.51/23] from ovn-kubernetes 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75 Error creating: pods "payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-23814d: serviceaccount "payload-processing-e2e-worker-w3-23814d" not found 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75 Created pod: payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz 61m Normal Pulling pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-23814d Scaled up replica set payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75 from 0 to 1 61m Normal Pulling pod/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Started pod/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt Container started 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt Add eth0 [10.134.0.54/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85 Created pod: payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-23814d Scaled up replica set payload-processing-e2e-worker-w3-23814d-6cb8b9dc85 from 0 to 1 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85 Error creating: pods "payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-23814d: serviceaccount "payload-processing-e2e-worker-w3-23814d" not found 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 Add eth0 [10.134.0.53/23] from ovn-kubernetes 61m Normal Created pod/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt Container created 61m Normal AddedInterface pod/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 Add eth0 [10.132.0.52/23] from ovn-kubernetes 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz Add eth0 [10.132.0.53/23] from ovn-kubernetes 61m Normal Started pod/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 Container started 61m Normal Pulling pod/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 61m Normal Created pod/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 Container created 61m Normal Pulled pod/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 Add eth0 [10.134.0.55/23] from ovn-kubernetes 61m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Readiness probe error: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Normal Pulled pod/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 9.388s (9.388s including waiting). Image size: 125353963 bytes. 61m Normal Started pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Container started 61m Normal Pulled pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 9.701s (9.701s including waiting). Image size: 125353963 bytes. 61m Normal Created pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Container created 61m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Readiness probe failed: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Normal Created pod/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz Container created 61m Normal Started pod/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz Container started 61m Normal Started pod/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 Container started 61m Normal Created pod/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 Container created 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 9.458s (9.458s including waiting). Image size: 125353963 bytes. 61m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Container created 61m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Container started 61m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Container started 61m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Container created 61m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Container started 61m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Container created 61m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Container started 61m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Container created 61m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Container created 61m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Container started 61m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Container started 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Container created 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused... 61m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Container created 61m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Container started 61m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused... 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc 60m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled up replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 1 to 2 60m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled up replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 1 to 2 60m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs 60m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Container started 60m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Container created 60m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Add eth0 [10.133.0.57/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Container started 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Container created 60m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Add eth0 [10.132.0.65/23] from ovn-kubernetes 60m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled up replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 1 to 2 60m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled up replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 1 to 2 60m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd Container started 60m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Add eth0 [10.133.0.59/23] from ovn-kubernetes 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn 60m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd Container created 60m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Container started 60m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Container created 60m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd Add eth0 [10.133.0.58/23] from ovn-kubernetes 60m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 60m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 60m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled up replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 1 to 2 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Add eth0 [10.132.0.68/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled up replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 1 to 2 60m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Container started 60m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Container created 60m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Container started 60m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Container created 60m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Add eth0 [10.132.0.69/23] from ovn-kubernetes 59m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 59m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled up replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 2 to 3 59m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Container started 59m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Add eth0 [10.134.0.65/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z 59m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Container created 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled up replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 1 to 2 59m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z Add eth0 [10.132.0.73/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z Container started 59m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z Container created 59m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled up replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 2 to 3 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk 59m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Container created 59m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd Container created 59m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk Container created 59m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk Container started 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled up replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 2 to 4 59m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Add eth0 [10.134.0.74/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv 59m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 59m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Container started 59m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk Add eth0 [10.134.0.72/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd Add eth0 [10.133.0.60/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled up replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 2 to 3 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Add eth0 [10.134.0.73/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Container started 59m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd Container started 59m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Container created 59m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 59m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 59m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 59m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled up replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 2 to 4 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq 59m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled up replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 3 to 5 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk 59m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq Add eth0 [10.134.0.76/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Add eth0 [10.132.0.85/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq Container started 59m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Container started 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w 59m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n Add eth0 [10.132.0.84/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n Container created 59m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n Container started 59m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Container created 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk 59m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Container started 59m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq Container created 59m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Add eth0 [10.133.0.62/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled up replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 2 to 4 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 59m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Container created 59m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w Add eth0 [10.134.0.77/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w Container created 59m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w Container started 59m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Add eth0 [10.133.0.63/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Container created 59m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Container started 58m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm 58m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs 58m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled up replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 4 to 5 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled up replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 3 to 5 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 58m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 Add eth0 [10.133.0.66/23] from ovn-kubernetes 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 5; reason: 58m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled up replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 4 to 5 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf 58m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm Add eth0 [10.133.0.65/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm Container created 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w 58m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs Container started 58m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs Container created 58m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs Add eth0 [10.132.0.90/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm Container started 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled up replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 2 to 4 58m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 Container started 58m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 Container created 58m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf Container started 58m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w Add eth0 [10.133.0.67/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Add eth0 [10.134.0.78/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf Container created 58m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf Add eth0 [10.132.0.91/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Container started 58m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w Container created 58m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w Container started 58m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Container created 58m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-2q74f 58m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 58m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-2q74f Container started 58m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-2q74f Container created 58m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-2q74f Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-2q74f Add eth0 [10.133.0.68/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb 58m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Container started 58m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled up replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 4 to 5 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled up replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 5 to 6 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled up replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 3 to 5 58m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Container started 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled up replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 4 to 7 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 58m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Container created 58m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Add eth0 [10.132.0.92/23] from ovn-kubernetes 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 5; reason: 58m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Add eth0 [10.134.0.81/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Container created 58m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Add eth0 [10.134.0.82/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx 58m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Container created 58m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Add eth0 [10.132.0.93/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Container started 58m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Container started 58m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Container created 58m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Add eth0 [10.132.0.94/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Container created 58m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Container started 58m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Container started 58m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Add eth0 [10.134.0.83/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr Add eth0 [10.132.0.95/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Container created 58m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr Container created 58m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr Container started 58m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q 58m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled up replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 5 to 6 58m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Container started 58m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled up replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 5 to 10 58m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Add eth0 [10.134.0.85/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 (combined from similar events): Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr 58m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Container created 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Created pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 58m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 Container created 58m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 Add eth0 [10.132.0.96/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt Add eth0 [10.132.0.97/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr Container created 58m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm Container started 58m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm Container created 58m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm Add eth0 [10.134.0.86/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 Container started 58m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr Container started 58m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Container started 58m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr Add eth0 [10.133.0.69/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Container created 58m Normal Pulled pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Add eth0 [10.134.0.87/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt Container started 58m Normal Created pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt Container created 58m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f (combined from similar events): Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc 58m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled up replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 6 to 10 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled up replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 5 to 6 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Created pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled up replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 5 to 8 58m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf Add eth0 [10.133.0.72/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw Add eth0 [10.134.0.89/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw Container created 58m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw Container started 58m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Add eth0 [10.134.0.90/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Container created 58m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Container started 58m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Container started 58m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Container created 58m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Container started 58m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Add eth0 [10.133.0.71/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf Container started 58m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf Container created 58m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Add eth0 [10.132.0.102/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 Add eth0 [10.133.0.73/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Container created 58m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 Container created 58m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 Container started 58m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Add eth0 [10.133.0.70/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Container started 58m Normal AddedInterface pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Add eth0 [10.132.0.101/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Container created 58m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Container started 58m Normal Created pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Container created 58m Normal Pulled pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled up replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 6 to 10 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 (combined from similar events): Created pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl 57m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Add eth0 [10.132.0.106/23] from ovn-kubernetes 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 57m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf Add eth0 [10.133.0.74/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf Container created 57m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Container created 57m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl Add eth0 [10.133.0.75/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 57m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr 57m Normal SuccessfulCreate replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 (combined from similar events): Created pod: e2e-worker-w0-23843f-openshift-default-54cf54885-88shr 57m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf Container started 57m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl Container created 57m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl Container started 57m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Container started 57m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled up replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 7 to 10 57m Normal Created pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Container created 57m Normal Pulled pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Container started 57m Normal AddedInterface pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Add eth0 [10.134.0.92/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Add eth0 [10.132.0.108/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Container started 57m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Container created 57m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Container started 57m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Add eth0 [10.133.0.76/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Container created 57m Normal Started pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Container started 57m Normal AddedInterface pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Add eth0 [10.134.0.93/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Container created 57m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Readiness probe error: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Readiness probe failed: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd 57m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm 57m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd Add eth0 [10.133.0.77/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled up replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 5 to 10 57m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Add eth0 [10.132.0.111/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 (combined from similar events): Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh 57m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 Add eth0 [10.134.0.97/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc (combined from similar events): Created pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled up replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 8 to 10 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 (combined from similar events): Created pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Created pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 57m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled up replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 6 to 10 57m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 Add eth0 [10.134.0.98/23] from ovn-kubernetes 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 57m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd Container created 57m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Container started 57m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Container created 57m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd Container started 57m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Add eth0 [10.132.0.112/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Add eth0 [10.133.0.78/23] from ovn-kubernetes 57m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 Container started 57m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 Container created 57m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Container created 57m Normal Created pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 Container created 57m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 Container started 57m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Add eth0 [10.133.0.79/23] from ovn-kubernetes 57m Normal Started pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Container started 57m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Add eth0 [10.132.0.113/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Add eth0 [10.134.0.99/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Add eth0 [10.134.0.100/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Container created 57m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Add eth0 [10.132.0.114/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Container created 57m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Container started 57m Normal Pulled pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Container started 57m Normal Pulled pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Readiness probe failed: Get "http://10.133.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Readiness probe error: Get "http://10.133.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Readiness probe error: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Container created 57m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Readiness probe failed: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Normal Created pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Container created 57m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Container started 57m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Container created 57m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Container started 57m Normal Created pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Container created 57m Normal Started pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Container started 57m Normal Started pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Container started 57m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Startup probe failed: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Startup probe error: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Startup probe failed: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Readiness probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Readiness probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Readiness probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Readiness probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Readiness probe failed: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Readiness probe error: Get "http://10.134.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Readiness probe failed: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Readiness probe error: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Readiness probe failed: Get "http://10.134.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Readiness probe error: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Readiness probe failed: Get "http://10.132.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Readiness probe error: Get "http://10.132.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Readiness probe error: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Readiness probe failed: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning Unhealthy pod/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 Readiness probe failed: timeout: failed to connect service "10.134.0.55:50052" within 1s: context deadline exceeded 56m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Readiness probe error: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Readiness probe failed: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Readiness probe error: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Readiness probe failed: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Readiness probe failed: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Readiness probe error: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal SuccessfulCreate replicaset/e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5 Created pod: e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5-2lbp5 56m Normal AddedLabel gateway/e2e-migrate-85be723d-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-85be723d-gw 56m Normal ScalingReplicaSet deployment/e2e-migrate-85be723d-gw-openshift-default Scaled up replica set e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5 from 0 to 1 56m Normal AddedInterface pod/e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5-2lbp5 Add eth0 [10.133.0.80/23] from ovn-kubernetes 56m Normal Started pod/e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5-2lbp5 Container started 56m Normal Created pod/e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5-2lbp5 Container created 56m Normal Pulled pod/e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5-2lbp5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal ScalingReplicaSet deployment/e2e-ait-f0303f0d-openshift-default Scaled up replica set e2e-ait-f0303f0d-openshift-default-5b5588d58f from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-ait-f0303f0d-openshift-default-5b5588d58f Created pod: e2e-ait-f0303f0d-openshift-default-5b5588d58f-c5sdh 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-85be723d Scaled up replica set payload-processing-e2e-migrate-85be723d-578cfdbb9c from 0 to 1 56m Normal AddedLabel gateway/e2e-ait-f0303f0d Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f0303f0d 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-85be723d-545664d4ff Error creating: pods "payload-pre-processing-e2e-migrate-85be723d-545664d4ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-85be723d: serviceaccount "payload-processing-e2e-migrate-85be723d" not found 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-85be723d Scaled up replica set payload-pre-processing-e2e-migrate-85be723d-545664d4ff from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-85be723d-578cfdbb9c Created pod: payload-processing-e2e-migrate-85be723d-578cfdbb9c-bh7qf 56m Warning FailedCreate replicaset/payload-processing-e2e-migrate-85be723d-578cfdbb9c Error creating: pods "payload-processing-e2e-migrate-85be723d-578cfdbb9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-85be723d: serviceaccount "payload-processing-e2e-migrate-85be723d" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-85be723d-545664d4ff Created pod: payload-pre-processing-e2e-migrate-85be723d-545664d4ff-7jldh 56m Normal Pulled pod/e2e-ait-f0303f0d-openshift-default-5b5588d58f-c5sdh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-ait-f0303f0d-openshift-default-5b5588d58f-c5sdh Container created 56m Normal Started pod/e2e-ait-f0303f0d-openshift-default-5b5588d58f-c5sdh Container started 56m Normal AddedInterface pod/e2e-ait-f0303f0d-openshift-default-5b5588d58f-c5sdh Add eth0 [10.132.0.125/23] from ovn-kubernetes 56m Normal Created pod/payload-pre-processing-e2e-migrate-85be723d-545664d4ff-7jldh Container created 56m Normal Pulled pod/payload-processing-e2e-migrate-85be723d-578cfdbb9c-bh7qf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Pulled pod/payload-pre-processing-e2e-migrate-85be723d-545664d4ff-7jldh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-85be723d-545664d4ff-7jldh Add eth0 [10.134.0.108/23] from ovn-kubernetes 56m Normal Created pod/payload-processing-e2e-migrate-85be723d-578cfdbb9c-bh7qf Container created 56m Normal Started pod/payload-processing-e2e-migrate-85be723d-578cfdbb9c-bh7qf Container started 56m Normal Started pod/payload-pre-processing-e2e-migrate-85be723d-545664d4ff-7jldh Container started 56m Normal AddedInterface pod/payload-processing-e2e-migrate-85be723d-578cfdbb9c-bh7qf Add eth0 [10.134.0.109/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-ait-1cc6560c-openshift-default-c5d595b6c Created pod: e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 56m Normal AddedLabel gateway/e2e-ait-1cc6560c Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1cc6560c 56m Normal ScalingReplicaSet deployment/e2e-ait-1cc6560c-openshift-default Scaled up replica set e2e-ait-1cc6560c-openshift-default-c5d595b6c from 0 to 1 56m Normal Pulled pod/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Warning FailedCreate replicaset/payload-processing-e2e-ait-f0303f0d-6f65d9f4c Error creating: pods "payload-processing-e2e-ait-f0303f0d-6f65d9f4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f0303f0d: serviceaccount "payload-processing-e2e-ait-f0303f0d" not found 56m Normal AddedInterface pod/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 Add eth0 [10.132.0.126/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-f0303f0d Scaled up replica set payload-pre-processing-e2e-ait-f0303f0d-7f59c48747 from 0 to 1 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747 Error creating: pods "payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f0303f0d: serviceaccount "payload-processing-e2e-ait-f0303f0d" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747 Created pod: payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-wntjb 56m Normal Created pod/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 Container created 56m Normal Started pod/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 Container started 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-f0303f0d Scaled up replica set payload-processing-e2e-ait-f0303f0d-6f65d9f4c from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-f0303f0d-6f65d9f4c Created pod: payload-processing-e2e-ait-f0303f0d-6f65d9f4c-z7c4z 56m Normal Pulled pod/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-wntjb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-wntjb Container started 56m Normal Pulled pod/payload-processing-e2e-ait-f0303f0d-6f65d9f4c-z7c4z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-ait-f0303f0d-6f65d9f4c-z7c4z Container created 56m Normal AddedInterface pod/payload-processing-e2e-ait-f0303f0d-6f65d9f4c-z7c4z Add eth0 [10.134.0.111/23] from ovn-kubernetes 56m Normal AddedInterface pod/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-wntjb Add eth0 [10.134.0.112/23] from ovn-kubernetes 56m Normal Started pod/payload-processing-e2e-ait-f0303f0d-6f65d9f4c-z7c4z Container started 56m Normal Created pod/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-wntjb Container created 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-1cc6560c-7f7db84869 Created pod: payload-processing-e2e-ait-1cc6560c-7f7db84869-wln55 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-1cc6560c Scaled up replica set payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5 from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-1cc6560c Scaled up replica set payload-processing-e2e-ait-1cc6560c-7f7db84869 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5 Created pod: payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-hq6bn 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5 Error creating: pods "payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1cc6560c: serviceaccount "payload-processing-e2e-ait-1cc6560c" not found 56m Warning FailedCreate replicaset/payload-processing-e2e-ait-1cc6560c-7f7db84869 Error creating: pods "payload-processing-e2e-ait-1cc6560c-7f7db84869-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1cc6560c: serviceaccount "payload-processing-e2e-ait-1cc6560c" not found 56m Normal AddedInterface pod/payload-processing-e2e-ait-1cc6560c-7f7db84869-wln55 Add eth0 [10.134.0.114/23] from ovn-kubernetes 56m Normal Created pod/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-hq6bn Container created 56m Normal Created pod/payload-processing-e2e-ait-1cc6560c-7f7db84869-wln55 Container created 56m Normal Started pod/payload-processing-e2e-ait-1cc6560c-7f7db84869-wln55 Container started 56m Normal Pulled pod/payload-processing-e2e-ait-1cc6560c-7f7db84869-wln55 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-hq6bn Add eth0 [10.133.0.81/23] from ovn-kubernetes 56m Normal Started pod/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-hq6bn Container started 56m Normal Pulled pod/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-hq6bn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-85be723d-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-85be723d-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f0303f0d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f0303f0d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1cc6560c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1cc6560c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Readiness probe error: Get "http://10.133.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Readiness probe failed: Get "http://10.133.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Readiness probe error: Get "http://10.133.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Readiness probe failed: Get "http://10.133.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Readiness probe error: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Readiness probe failed: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-85be723d-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-85be723d-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f0303f0d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f0303f0d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-bhgqx 55m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 55m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 55m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Add eth0 [10.132.0.133/23] from ovn-kubernetes 55m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.663s (3.663s including waiting). Image size: 185476477 bytes. 55m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Container created 55m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Container started 55m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Readiness probe error: HTTP probe failed with statuscode: 503... 55m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Readiness probe failed: HTTP probe failed with statuscode: 503 55m Normal Killing pod/payload-processing-e2e-migrate-85be723d-578cfdbb9c-bh7qf Stopping container payload-processing 55m Normal Killing pod/payload-pre-processing-e2e-migrate-85be723d-545664d4ff-7jldh Stopping container payload-pre-processing 55m Normal Killing pod/payload-processing-e2e-ait-f0303f0d-6f65d9f4c-z7c4z Stopping container payload-processing 55m Normal Killing pod/payload-pre-processing-e2e-ait-f0303f0d-7f59c48747-wntjb Stopping container payload-pre-processing 55m Normal Killing pod/e2e-migrate-85be723d-gw-openshift-default-76d9b68fb5-2lbp5 Stopping container istio-proxy 55m Normal SuccessfulCreate replicaset/e2e-ait-63eab93e-openshift-default-6d4c965c9c Created pod: e2e-ait-63eab93e-openshift-default-6d4c965c9c-lwwjn 55m Normal AddedLabel gateway/e2e-ait-63eab93e Added label istio.io/rev=openshift-gateway to gateway e2e-ait-63eab93e 55m Normal ScalingReplicaSet deployment/e2e-ait-63eab93e-openshift-default Scaled up replica set e2e-ait-63eab93e-openshift-default-6d4c965c9c from 0 to 1 55m Normal Killing pod/payload-processing-e2e-ait-1cc6560c-7f7db84869-wln55 Stopping container payload-processing 55m Normal AddedInterface pod/e2e-ait-63eab93e-openshift-default-6d4c965c9c-lwwjn Add eth0 [10.133.0.82/23] from ovn-kubernetes 55m Normal Started pod/e2e-ait-63eab93e-openshift-default-6d4c965c9c-lwwjn Container started 55m Normal Created pod/e2e-ait-63eab93e-openshift-default-6d4c965c9c-lwwjn Container created 55m Normal Killing pod/payload-pre-processing-e2e-ait-1cc6560c-58795b5bc5-hq6bn Stopping container payload-pre-processing 55m Normal Pulled pod/e2e-ait-63eab93e-openshift-default-6d4c965c9c-lwwjn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Killing pod/e2e-ait-f0303f0d-openshift-default-5b5588d58f-c5sdh Stopping container istio-proxy 55m Normal AddedLabel gateway/e2e-derive-0b9ff685 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-0b9ff685 55m Normal SuccessfulCreate replicaset/e2e-derive-0b9ff685-openshift-default-648b65bf89 Created pod: e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt 55m Normal ScalingReplicaSet deployment/e2e-derive-0b9ff685-openshift-default Scaled up replica set e2e-derive-0b9ff685-openshift-default-648b65bf89 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff Created pod: payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-5pq8p 55m Normal Created pod/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt Container created 55m Normal Pulled pod/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt Add eth0 [10.133.0.83/23] from ovn-kubernetes 55m Warning FailedCreate replicaset/payload-processing-e2e-ait-63eab93e-5bdf4b59f9 Error creating: pods "payload-processing-e2e-ait-63eab93e-5bdf4b59f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-63eab93e: serviceaccount "payload-processing-e2e-ait-63eab93e" not found 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-63eab93e Scaled up replica set payload-processing-e2e-ait-63eab93e-5bdf4b59f9 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-63eab93e-5bdf4b59f9 Created pod: payload-processing-e2e-ait-63eab93e-5bdf4b59f9-bsxqw 55m Normal Started pod/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt Container started 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-63eab93e Scaled up replica set payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff Error creating: pods "payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-63eab93e: serviceaccount "payload-processing-e2e-ait-63eab93e" not found 55m Normal Pulled pod/payload-processing-e2e-ait-63eab93e-5bdf4b59f9-bsxqw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-processing-e2e-ait-63eab93e-5bdf4b59f9-bsxqw Add eth0 [10.134.0.118/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-5pq8p Add eth0 [10.134.0.117/23] from ovn-kubernetes 55m Normal Pulled pod/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-5pq8p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-processing-e2e-ait-63eab93e-5bdf4b59f9-bsxqw Container started 55m Normal Started pod/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-5pq8p Container started 55m Normal Created pod/payload-processing-e2e-ait-63eab93e-5bdf4b59f9-bsxqw Container created 55m Normal Created pod/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-5pq8p Container created 55m Normal Killing pod/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 Stopping container istio-proxy 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-0b9ff685 Scaled up replica set payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79 from 0 to 1 55m Warning FailedCreate replicaset/payload-processing-e2e-derive-0b9ff685-b95f54655 Error creating: pods "payload-processing-e2e-derive-0b9ff685-b95f54655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-0b9ff685: serviceaccount "payload-processing-e2e-derive-0b9ff685" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-0b9ff685-b95f54655 Created pod: payload-processing-e2e-derive-0b9ff685-b95f54655-qk54m 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79 Error creating: pods "payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-0b9ff685: serviceaccount "payload-processing-e2e-derive-0b9ff685" not found 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-0b9ff685 Scaled up replica set payload-processing-e2e-derive-0b9ff685-b95f54655 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79 Created pod: payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-wp72k 55m Warning Unhealthy pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Readiness probe failed: timeout: failed to connect service "10.132.0.51:50052" within 1s: context deadline exceeded 55m Normal Started pod/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-wp72k Container started 55m Normal Created pod/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-wp72k Container created 55m Normal Started pod/payload-processing-e2e-derive-0b9ff685-b95f54655-qk54m Container started 55m Normal Created pod/payload-processing-e2e-derive-0b9ff685-b95f54655-qk54m Container created 55m Normal AddedInterface pod/payload-processing-e2e-derive-0b9ff685-b95f54655-qk54m Add eth0 [10.134.0.119/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-wp72k Add eth0 [10.134.0.120/23] from ovn-kubernetes 55m Normal Pulled pod/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-wp72k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Pulled pod/payload-processing-e2e-derive-0b9ff685-b95f54655-qk54m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Warning ProbeError pod/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 Readiness probe error: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/e2e-ait-1cc6560c-openshift-default-c5d595b6c-7mpf7 Readiness probe failed: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-0b9ff685-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-0b9ff685-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal Killing pod/payload-processing-e2e-ait-63eab93e-5bdf4b59f9-bsxqw Stopping container payload-processing 54m Normal Killing pod/payload-pre-processing-e2e-ait-63eab93e-8596d6f7ff-5pq8p Stopping container payload-pre-processing 54m Normal Killing pod/payload-pre-processing-e2e-derive-0b9ff685-85b47b7c79-wp72k Stopping container payload-pre-processing 54m Normal Killing pod/payload-processing-e2e-derive-0b9ff685-b95f54655-qk54m Stopping container payload-processing 54m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Readiness probe failed: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Readiness probe failed: Get "http://10.132.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Readiness probe error: Get "http://10.132.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-63eab93e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-63eab93e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-cd9729-openshift-default Scaled up replica set e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869 from 0 to 1 54m Normal AddedLabel gateway/e2e-shared-a-w1-cd9729 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-cd9729 54m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869 Created pod: e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll 54m Normal Killing pod/e2e-ait-63eab93e-openshift-default-6d4c965c9c-lwwjn Stopping container istio-proxy 54m Normal AddedInterface pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Add eth0 [10.133.0.84/23] from ovn-kubernetes 54m Normal Killing pod/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt Stopping container istio-proxy 54m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-a891ce-openshift-default Scaled up replica set e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 from 0 to 1 54m Warning Unhealthy pod/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt Readiness probe failed: HTTP probe failed with statuscode: 503 54m Warning ProbeError pod/e2e-derive-0b9ff685-openshift-default-648b65bf89-k5qjt Readiness probe error: HTTP probe failed with statuscode: 503... 54m Normal AddedLabel gateway/e2e-shared-a-w6-a891ce Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-a891ce 54m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 Created pod: e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd 54m Normal AddedInterface pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd Add eth0 [10.134.0.123/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-cd9729 Scaled up replica set payload-processing-e2e-shared-a-w1-cd9729-774576d6f5 from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc Created pod: payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-cd9729 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5 Error creating: pods "payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-cd9729: serviceaccount "payload-processing-e2e-shared-a-w1-cd9729" not found 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc Error creating: pods "payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-cd9729: serviceaccount "payload-processing-e2e-shared-a-w1-cd9729" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5 Created pod: payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 Add eth0 [10.134.0.124/23] from ovn-kubernetes 54m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w Add eth0 [10.134.0.125/23] from ovn-kubernetes 54m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 Container created 54m Normal Created pod/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w Container created 54m Normal Pulled pod/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w Container started 54m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 Container started 54m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-a891ce Scaled up replica set payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7 from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-a891ce Scaled up replica set payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55 from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55 Created pod: payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-a891ce: serviceaccount "payload-processing-e2e-shared-a-w6-a891ce" not found 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7 Error creating: pods "payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-a891ce: serviceaccount "payload-processing-e2e-shared-a-w6-a891ce" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7 Created pod: payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs Add eth0 [10.134.0.126/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj Add eth0 [10.134.0.127/23] from ovn-kubernetes 54m Normal Created pod/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj Container created 54m Normal Pulled pod/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj Container started 54m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs Container created 54m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs Container started 54m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7 Created pod: e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 54m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-435728-openshift-default Scaled up replica set e2e-shared-b-w1-435728-openshift-default-5f778d7cf7 from 0 to 1 54m Normal AddedLabel gateway/e2e-shared-b-w1-435728 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-435728 54m Normal AddedInterface pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 Add eth0 [10.133.0.85/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 Created pod: e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl 54m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e1d060-openshift-default Scaled up replica set e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 from 0 to 1 54m Normal AddedInterface pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl Add eth0 [10.133.0.86/23] from ovn-kubernetes 54m Normal AddedLabel gateway/e2e-shared-b-w6-e1d060 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-e1d060 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-435728 Scaled up replica set payload-processing-e2e-shared-b-w1-435728-7b9576b5ff from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff Error creating: pods "payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-435728: serviceaccount "payload-processing-e2e-shared-b-w1-435728" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff Created pod: payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-435728 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559 from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559 Created pod: payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-435728: serviceaccount "payload-processing-e2e-shared-b-w1-435728" not found 54m Normal Started pod/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 Add eth0 [10.134.0.130/23] from ovn-kubernetes 54m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 Container started 54m Normal Pulled pod/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 Container created 54m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 Add eth0 [10.134.0.131/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 Container created 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-e1d060 Scaled up replica set payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5 from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5 Error creating: pods "payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-e1d060: serviceaccount "payload-processing-e2e-shared-b-w6-e1d060" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5 Created pod: payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-e1d060 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f Created pod: payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f Error creating: pods "payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-e1d060: serviceaccount "payload-processing-e2e-shared-b-w6-e1d060" not found 54m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk Add eth0 [10.134.0.133/23] from ovn-kubernetes 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr Add eth0 [10.134.0.132/23] from ovn-kubernetes 54m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr Container created 54m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr Container started 54m Normal Created pod/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk Container created 54m Normal Started pod/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk Container started 54m Normal Pulled pod/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw 53m Normal ScalingReplicaSet deployment/e2e-mt-67adab47-openshift-default Scaled up replica set e2e-mt-67adab47-openshift-default-84fb98c68f from 0 to 1 53m Normal AddedLabel gateway/e2e-mt-67adab47 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-67adab47 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Readiness probe failed: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Readiness probe failed: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw Add eth0 [10.132.0.140/23] from ovn-kubernetes 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-67adab47-998479859 Error creating: pods "payload-pre-processing-e2e-mt-67adab47-998479859-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-67adab47: serviceaccount "payload-processing-e2e-mt-67adab47" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-67adab47-7ddbcc466c Created pod: payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 53m Warning FailedCreate replicaset/payload-processing-e2e-mt-67adab47-7ddbcc466c Error creating: pods "payload-processing-e2e-mt-67adab47-7ddbcc466c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-67adab47: serviceaccount "payload-processing-e2e-mt-67adab47" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-67adab47-998479859 Created pod: payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-67adab47 Scaled up replica set payload-pre-processing-e2e-mt-67adab47-998479859 from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-67adab47 Scaled up replica set payload-processing-e2e-mt-67adab47-7ddbcc466c from 0 to 1 53m Normal Pulled pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 Container created 53m Warning Unhealthy pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Readiness probe failed: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Normal Started pod/payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v Container started 53m Normal AddedInterface pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 Add eth0 [10.134.0.136/23] from ovn-kubernetes 53m Normal Created pod/payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v Container created 53m Normal Pulled pod/payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Warning ProbeError pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Readiness probe error: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Normal AddedInterface pod/payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v Add eth0 [10.133.0.88/23] from ovn-kubernetes 53m Normal Started pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 Container started 53m Normal Created pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Container created 53m Normal Started pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 Container started 53m Normal Pulled pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Container started 53m Normal Created pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 Container created 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-cd9729-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-cd9729-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning Unhealthy pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 53m Warning ProbeError pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-435728-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-435728-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Created pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl Container created 53m Normal Pulled pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl Container started 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-cd9729-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-cd9729-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning Unhealthy pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 53m Warning ProbeError pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-435728-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-435728-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal AddedLabel gateway/e2e-mt-db61066b Added label istio.io/rev=openshift-gateway to gateway e2e-mt-db61066b 52m Normal ScalingReplicaSet deployment/e2e-mt-db61066b-openshift-default Scaled up replica set e2e-mt-db61066b-openshift-default-f9db4cc6f from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-mt-db61066b-openshift-default-f9db4cc6f Created pod: e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 52m Normal AddedInterface pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Add eth0 [10.132.0.142/23] from ovn-kubernetes 52m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw Container started 52m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw Container created 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-67adab47-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-67adab47-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c Created pod: payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-cf2lr 52m Normal ScalingReplicaSet deployment/e2e-api-a-9e323f-openshift-default Scaled up replica set e2e-api-a-9e323f-openshift-default-75fcf97f65 from 0 to 1 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c Error creating: pods "payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-db61066b: serviceaccount "payload-processing-e2e-mt-db61066b" not found 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-db61066b Scaled up replica set payload-processing-e2e-mt-db61066b-5cbb466f49 from 0 to 1 52m Warning FailedCreate replicaset/payload-processing-e2e-mt-db61066b-5cbb466f49 Error creating: pods "payload-processing-e2e-mt-db61066b-5cbb466f49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-db61066b: serviceaccount "payload-processing-e2e-mt-db61066b" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-db61066b-5cbb466f49 Created pod: payload-processing-e2e-mt-db61066b-5cbb466f49-5zjw7 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-db61066b Scaled up replica set payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c from 0 to 1 52m Normal AddedLabel gateway/e2e-api-a-9e323f Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-9e323f 52m Normal SuccessfulCreate replicaset/e2e-api-a-9e323f-openshift-default-75fcf97f65 Created pod: e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl 52m Normal Pulled pod/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-cf2lr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-cf2lr Container created 52m Normal AddedInterface pod/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-cf2lr Add eth0 [10.134.0.137/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-mt-db61066b-5cbb466f49-5zjw7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-mt-db61066b-5cbb466f49-5zjw7 Container created 52m Normal Started pod/payload-processing-e2e-mt-db61066b-5cbb466f49-5zjw7 Container started 52m Normal AddedInterface pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Add eth0 [10.133.0.94/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-cf2lr Container started 52m Normal AddedInterface pod/payload-processing-e2e-mt-db61066b-5cbb466f49-5zjw7 Add eth0 [10.134.0.138/23] from ovn-kubernetes 52m Normal AddedLabel gateway/e2e-shared-a-w2-4bac5a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-4bac5a 52m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-4bac5a-openshift-default Scaled up replica set e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647 from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647 Created pod: e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc 52m Normal AddedInterface pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Add eth0 [10.132.0.143/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused 52m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9 Created pod: payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-kjwgx 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9 Error creating: pods "payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-9e323f: serviceaccount "payload-processing-e2e-api-a-9e323f" not found 52m Warning FailedCreate replicaset/payload-processing-e2e-api-a-9e323f-778fbb59d7 Error creating: pods "payload-processing-e2e-api-a-9e323f-778fbb59d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-9e323f: serviceaccount "payload-processing-e2e-api-a-9e323f" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-9e323f-778fbb59d7 Created pod: payload-processing-e2e-api-a-9e323f-778fbb59d7-49m62 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-9e323f Scaled up replica set payload-pre-processing-e2e-api-a-9e323f-85744fc5f9 from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-9e323f Scaled up replica set payload-processing-e2e-api-a-9e323f-778fbb59d7 from 0 to 1 52m Normal Pulled pod/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-kjwgx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-api-a-9e323f-778fbb59d7-49m62 Container created 52m Normal Started pod/payload-processing-e2e-api-a-9e323f-778fbb59d7-49m62 Container started 52m Normal Started pod/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-kjwgx Container started 52m Normal Created pod/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-kjwgx Container created 52m Normal AddedInterface pod/payload-processing-e2e-api-a-9e323f-778fbb59d7-49m62 Add eth0 [10.134.0.140/23] from ovn-kubernetes 52m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-kjwgx Add eth0 [10.134.0.139/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-api-a-9e323f-778fbb59d7-49m62 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc Created pod: payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-4bac5a Scaled up replica set payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc from 0 to 1 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc Error creating: pods "payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-4bac5a: serviceaccount "payload-processing-e2e-shared-a-w2-4bac5a" not found 52m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc Error creating: pods "payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-4bac5a: serviceaccount "payload-processing-e2e-shared-a-w2-4bac5a" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc Created pod: payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-4bac5a Scaled up replica set payload-processing-e2e-shared-a-w2-4bac5a-78664677cc from 0 to 1 52m Normal Pulled pod/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 Container created 52m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd Add eth0 [10.133.0.95/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 Container started 52m Normal Created pod/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd Container created 52m Normal Started pod/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd Container started 52m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 Add eth0 [10.134.0.141/23] from ovn-kubernetes 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-e1d060-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-e1d060-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-67adab47-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-67adab47-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-4bac5a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-4bac5a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal Killing pod/payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v Stopping container payload-pre-processing 51m Normal Killing pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 Stopping container payload-processing 51m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw 51m Normal AddedLabel gateway/e2e-api-b-d4b21c Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-d4b21c 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-db61066b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-db61066b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal ScalingReplicaSet deployment/e2e-api-b-d4b21c-openshift-default Scaled up replica set e2e-api-b-d4b21c-openshift-default-5f69685d47 from 0 to 1 51m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Add eth0 [10.133.0.97/23] from ovn-kubernetes 51m Normal AddedLabel gateway/e2e-shared-b-w2-ae2e60 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-ae2e60 51m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae2e60-openshift-default Scaled up replica set e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 51m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 Add eth0 [10.132.0.148/23] from ovn-kubernetes 51m Warning ProbeError pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Readiness probe error: Get "http://10.132.0.142:15021/healthz/ready": read tcp 10.132.0.2:42374->10.132.0.142:15021: read: connection reset by peer... 51m Warning Unhealthy pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Readiness probe failed: Get "http://10.132.0.142:15021/healthz/ready": read tcp 10.132.0.2:42374->10.132.0.142:15021: read: connection reset by peer 51m Normal Pulled pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Container started 51m Normal Created pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Container created 51m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Deleted pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7 Error creating: pods "payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d4b21c: serviceaccount "payload-processing-e2e-api-b-d4b21c" not found 51m Normal SuccessfulDelete replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Deleted pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-chr6z 51m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-8crc9 Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-d4b21c Scaled up replica set payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7 from 0 to 1 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 7; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled down replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 10 to 7 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-d4b21c Scaled up replica set payload-processing-e2e-api-b-d4b21c-764698f6f from 0 to 1 51m Normal SuccessfulDelete replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Deleted pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-c9btf 51m Warning FailedCreate replicaset/payload-processing-e2e-api-b-d4b21c-764698f6f Error creating: pods "payload-processing-e2e-api-b-d4b21c-764698f6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d4b21c: serviceaccount "payload-processing-e2e-api-b-d4b21c" not found 51m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-rj87s Add eth0 [10.133.0.98/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-d4b21c-764698f6f Created pod: payload-processing-e2e-api-b-d4b21c-764698f6f-dt2q6 51m Normal Pulled pod/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-rj87s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-api-b-d4b21c-764698f6f-dt2q6 Add eth0 [10.133.0.99/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7 Created pod: payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-rj87s 51m Normal Started pod/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-rj87s Container started 51m Normal Started pod/payload-processing-e2e-api-b-d4b21c-764698f6f-dt2q6 Container started 51m Normal Created pod/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-rj87s Container created 51m Normal Created pod/payload-processing-e2e-api-b-d4b21c-764698f6f-dt2q6 Container created 51m Normal Pulled pod/payload-processing-e2e-api-b-d4b21c-764698f6f-dt2q6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-ae2e60 Scaled up replica set payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9 Created pod: payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 51m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9 Error creating: pods "payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-ae2e60: serviceaccount "payload-processing-e2e-shared-b-w2-ae2e60" not found 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f Error creating: pods "payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-ae2e60: serviceaccount "payload-processing-e2e-shared-b-w2-ae2e60" not found 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f Created pod: payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-ae2e60 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f from 0 to 1 51m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 Add eth0 [10.134.0.145/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj Container started 51m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj Container created 51m Normal Started pod/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 Container started 51m Normal Created pod/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 Container created 51m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj Add eth0 [10.134.0.144/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Startup probe failed: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused 51m Warning ProbeError pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Startup probe error: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused... 51m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-l4p8z 51m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 51m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-l4p8z Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 51m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-l4p8z Container created 51m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-l4p8z Container started 51m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-l4p8z Add eth0 [10.132.0.150/23] from ovn-kubernetes 51m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-l4p8z Readiness probe error: HTTP probe failed with statuscode: 503... 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-67adab47-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-l4p8z Readiness probe failed: HTTP probe failed with statuscode: 503 51m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt 51m Normal ScalingReplicaSet deployment/e2e-mt-67adab47-openshift-default Scaled up replica set e2e-mt-67adab47-openshift-default-84fb98c68f from 1 to 2 51m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Container started 51m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Add eth0 [10.133.0.100/23] from ovn-kubernetes 51m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Container created 51m Warning Unhealthy pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-nnms7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 51m Warning Unhealthy pod/payload-pre-processing-e2e-mt-67adab47-998479859-kmf5v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 51m Warning FailedCreate replicaset/payload-processing-e2e-mt-67adab47-7ddbcc466c Error creating: pods "payload-processing-e2e-mt-67adab47-7ddbcc466c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-67adab47: serviceaccount "payload-processing-e2e-mt-67adab47" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-67adab47 Scaled up replica set payload-processing-e2e-mt-67adab47-7ddbcc466c from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-67adab47 Scaled up replica set payload-pre-processing-e2e-mt-67adab47-998479859 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-67adab47-998479859 Created pod: payload-pre-processing-e2e-mt-67adab47-998479859-c8zq5 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-67adab47-998479859 Error creating: pods "payload-pre-processing-e2e-mt-67adab47-998479859-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-67adab47: serviceaccount "payload-processing-e2e-mt-67adab47" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-67adab47-7ddbcc466c Created pod: payload-processing-e2e-mt-67adab47-7ddbcc466c-n2bkz 51m Normal AddedInterface pod/payload-pre-processing-e2e-mt-67adab47-998479859-c8zq5 Add eth0 [10.133.0.101/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-mt-67adab47-998479859-c8zq5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-n2bkz Container started 51m Normal Started pod/payload-pre-processing-e2e-mt-67adab47-998479859-c8zq5 Container started 51m Normal AddedInterface pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-n2bkz Add eth0 [10.133.0.102/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-n2bkz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-n2bkz Container created 51m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 51m Normal Created pod/payload-pre-processing-e2e-mt-67adab47-998479859-c8zq5 Container created 51m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-db61066b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-db61066b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled down replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 10 to 9 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 9; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Deleted pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-cr28v 51m Normal Created pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Container created 51m Normal Pulled pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Container started 51m Normal SuccessfulCreate replicaset/e2e-api-a-e73256-openshift-default-dfc9897d9 Created pod: e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk 51m Normal ScalingReplicaSet deployment/e2e-api-a-e73256-openshift-default Scaled up replica set e2e-api-a-e73256-openshift-default-dfc9897d9 from 0 to 1 51m Normal AddedLabel gateway/e2e-api-a-e73256 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-e73256 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled down replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 9 to 8 51m Normal AddedInterface pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Add eth0 [10.132.0.152/23] from ovn-kubernetes 51m Normal SuccessfulDelete replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Deleted pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled down replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 10 to 7 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 8; reason: All metrics below target 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 7; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Deleted pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Deleted pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-9zkgd 51m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Deleted pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd 51m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-ncxxd Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-qzm84 Stopping container istio-proxy 51m Warning ProbeError pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-jg8gt Readiness probe failed: HTTP probe failed with statuscode: 503 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-e73256-b9d6b9887 Created pod: payload-pre-processing-e2e-api-a-e73256-b9d6b9887-62lj9 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-e73256 Scaled up replica set payload-pre-processing-e2e-api-a-e73256-b9d6b9887 from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-api-a-e73256-57b5c8849f Error creating: pods "payload-processing-e2e-api-a-e73256-57b5c8849f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e73256: serviceaccount "payload-processing-e2e-api-a-e73256" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-e73256-57b5c8849f Created pod: payload-processing-e2e-api-a-e73256-57b5c8849f-n9qzl 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-e73256-b9d6b9887 Error creating: pods "payload-pre-processing-e2e-api-a-e73256-b9d6b9887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e73256: serviceaccount "payload-processing-e2e-api-a-e73256" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-e73256 Scaled up replica set payload-processing-e2e-api-a-e73256-57b5c8849f from 0 to 1 50m Normal Started pod/payload-processing-e2e-api-a-e73256-57b5c8849f-n9qzl Container started 50m Normal Pulled pod/payload-processing-e2e-api-a-e73256-57b5c8849f-n9qzl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-api-a-e73256-b9d6b9887-62lj9 Container started 50m Normal Pulled pod/payload-pre-processing-e2e-api-a-e73256-b9d6b9887-62lj9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-api-a-e73256-57b5c8849f-n9qzl Container created 50m Normal Created pod/payload-pre-processing-e2e-api-a-e73256-b9d6b9887-62lj9 Container created 50m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-e73256-b9d6b9887-62lj9 Add eth0 [10.134.0.148/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-processing-e2e-api-a-e73256-57b5c8849f-n9qzl Add eth0 [10.133.0.104/23] from ovn-kubernetes 50m Normal Killing pod/payload-pre-processing-e2e-api-a-9e323f-85744fc5f9-kjwgx Stopping container payload-pre-processing 50m Normal Killing pod/payload-processing-e2e-api-a-9e323f-778fbb59d7-49m62 Stopping container payload-processing 50m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Created pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c 50m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-fd1c91-openshift-default Scaled up replica set e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff from 0 to 1 50m Normal AddedLabel gateway/e2e-shared-a-w0-fd1c91 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-fd1c91 50m Normal AddedInterface pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Add eth0 [10.133.0.105/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465 Created pod: payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-fd1c91 Scaled up replica set payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465 from 0 to 1 50m Normal SuccessfulDelete replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Deleted pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-fd1c91 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9 from 0 to 1 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 7; reason: All metrics below target 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default Scaled down replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 8 to 7 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9 Created pod: payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-fd1c91: serviceaccount "payload-processing-e2e-shared-a-w0-fd1c91" not found 50m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465 Error creating: pods "payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-fd1c91: serviceaccount "payload-processing-e2e-shared-a-w0-fd1c91" not found 50m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd Add eth0 [10.133.0.107/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 Add eth0 [10.133.0.108/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 Container created 50m Normal Started pod/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 Container started 50m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Stopping container istio-proxy 50m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd Container started 50m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd Container created 50m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ckfzm Readiness probe failed: HTTP probe failed with statuscode: 503 50m Normal SuccessfulCreate replicaset/e2e-inf-a-03dfee-openshift-default-5ffbd99665 Created pod: e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 50m Normal ScalingReplicaSet deployment/e2e-inf-a-03dfee-openshift-default Scaled up replica set e2e-inf-a-03dfee-openshift-default-5ffbd99665 from 0 to 1 50m Normal AddedLabel gateway/e2e-inf-a-03dfee Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-03dfee 50m Normal Pulled pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Container started 50m Normal Created pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Container created 50m Normal AddedInterface pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Add eth0 [10.132.0.154/23] from ovn-kubernetes 50m Warning Unhealthy pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Readiness probe failed: Get "http://10.133.0.94:15021/healthz/ready": read tcp 10.133.0.2:45998->10.133.0.94:15021: read: connection reset by peer 50m Warning ProbeError pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Readiness probe error: Get "http://10.133.0.94:15021/healthz/ready": read tcp 10.133.0.2:45998->10.133.0.94:15021: read: connection reset by peer... 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-03dfee Scaled up replica set payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-03dfee-6fb7d4f559 Error creating: pods "payload-processing-e2e-inf-a-03dfee-6fb7d4f559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-03dfee: serviceaccount "payload-processing-e2e-inf-a-03dfee" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-03dfee-6fb7d4f559 Created pod: payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-03dfee Scaled up replica set payload-processing-e2e-inf-a-03dfee-6fb7d4f559 from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c Error creating: pods "payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-03dfee: serviceaccount "payload-processing-e2e-inf-a-03dfee" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c Created pod: payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk 50m Normal Killing pod/payload-processing-e2e-mt-67adab47-7ddbcc466c-n2bkz Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-mt-67adab47-998479859-c8zq5 Stopping container payload-pre-processing 50m Normal Created pod/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk Container created 50m Normal Started pod/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk Container started 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk Add eth0 [10.134.0.149/23] from ovn-kubernetes 50m Normal Created pod/payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v Container created 50m Normal Pulled pod/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Pulled pod/payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v Add eth0 [10.134.0.150/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v Container started 50m Warning ProbeError pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-9e323f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-9e323f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-4bac5a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-4bac5a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-d4b21c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-api-b-d4b21c-openshift-default Scaled up replica set e2e-api-b-d4b21c-openshift-default-5f69685d47 from 1 to 4 50m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc 50m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Add eth0 [10.132.0.156/23] from ovn-kubernetes 50m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Container started 50m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc Add eth0 [10.133.0.110/23] from ovn-kubernetes 50m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc Container started 50m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Container created 50m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc Container created 50m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z 50m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z Container started 50m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z Container created 50m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z Add eth0 [10.134.0.152/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-mt-67adab47-openshift-default Scaled up replica set e2e-mt-67adab47-openshift-default-84fb98c68f from 2 to 4 50m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-67adab47-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 50m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Container created 50m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Container started 50m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Add eth0 [10.133.0.112/23] from ovn-kubernetes 50m Normal Killing pod/e2e-api-a-9e323f-openshift-default-75fcf97f65-bcfzl Stopping container istio-proxy 50m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Container created 50m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Readiness probe error: Get "http://10.133.0.97:15021/healthz/ready": read tcp 10.133.0.2:39484->10.133.0.97:15021: read: connection reset by peer... 50m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Readiness probe failed: Get "http://10.133.0.97:15021/healthz/ready": read tcp 10.133.0.2:39484->10.133.0.97:15021: read: connection reset by peer 50m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Container started 50m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Add eth0 [10.134.0.153/23] from ovn-kubernetes 50m Normal SuccessfulDelete replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Deleted pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg 50m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 Stopping container istio-proxy 50m Normal AddedLabel gateway/e2e-api-b-1c0c1d Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-1c0c1d 50m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled down replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 10 to 7 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 7; reason: All metrics below target 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Deleted pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-drnb4 50m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Container started 50m Normal SuccessfulDelete replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Deleted pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj 50m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Stopping container istio-proxy 50m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Container created 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Deleted pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-7qhq2 50m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Deleted pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-b6fwc 50m Normal SuccessfulDelete replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Deleted pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled down replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 10 to 7 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 7; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4hh2s Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-api-b-1c0c1d-openshift-default Scaled up replica set e2e-api-b-1c0c1d-openshift-default-857c68594c from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-api-b-1c0c1d-openshift-default-857c68594c Created pod: e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 50m Normal Killing pod/payload-processing-e2e-mt-db61066b-5cbb466f49-5zjw7 Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-mt-db61066b-7fd4c8cb6c-cf2lr Stopping container payload-pre-processing 50m Normal AddedInterface pod/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 Add eth0 [10.133.0.113/23] from ovn-kubernetes 50m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Startup probe error: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Startup probe failed: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc Startup probe failed: Get "http://10.133.0.110:15021/healthz/ready": dial tcp 10.133.0.110:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused 50m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-4s2jg Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc Startup probe error: Get "http://10.133.0.110:15021/healthz/ready": dial tcp 10.133.0.110:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-vc4dj Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Startup probe error: Get "http://10.133.0.112:15021/healthz/ready": dial tcp 10.133.0.112:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 50m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Startup probe failed: Get "http://10.133.0.112:15021/healthz/ready": dial tcp 10.133.0.112:15021: connect: connection refused 50m Warning FailedCreate replicaset/payload-processing-e2e-api-b-1c0c1d-758fdd696f Error creating: pods "payload-processing-e2e-api-b-1c0c1d-758fdd696f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-1c0c1d: serviceaccount "payload-processing-e2e-api-b-1c0c1d" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7 Created pod: payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-c79ts 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-1c0c1d-758fdd696f Created pod: payload-processing-e2e-api-b-1c0c1d-758fdd696f-ff9zq 50m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-c79ts Add eth0 [10.133.0.114/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-1c0c1d Scaled up replica set payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7 from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7 Error creating: pods "payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-1c0c1d: serviceaccount "payload-processing-e2e-api-b-1c0c1d" not found 50m Normal Pulled pod/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-c79ts Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-1c0c1d Scaled up replica set payload-processing-e2e-api-b-1c0c1d-758fdd696f from 0 to 1 50m Normal Created pod/payload-processing-e2e-api-b-1c0c1d-758fdd696f-ff9zq Container created 50m Normal Started pod/payload-processing-e2e-api-b-1c0c1d-758fdd696f-ff9zq Container started 50m Normal Created pod/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-c79ts Container created 50m Normal Killing pod/payload-processing-e2e-api-b-d4b21c-764698f6f-dt2q6 Stopping container payload-processing 50m Normal Pulled pod/payload-processing-e2e-api-b-1c0c1d-758fdd696f-ff9zq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-api-b-1c0c1d-758fdd696f-ff9zq Add eth0 [10.132.0.158/23] from ovn-kubernetes 50m Normal Started pod/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-c79ts Container started 50m Normal Killing pod/payload-pre-processing-e2e-api-b-d4b21c-6bf64967d7-rj87s Stopping container payload-pre-processing 50m Normal ScalingReplicaSet deployment/e2e-mt-67adab47-openshift-default Scaled up replica set e2e-mt-67adab47-openshift-default-84fb98c68f from 4 to 8 50m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg 50m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 50m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-67adab47-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf 50m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Add eth0 [10.134.0.154/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Add eth0 [10.132.0.159/23] from ovn-kubernetes 50m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Container started 50m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Container started 50m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Container created 50m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Container created 50m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Add eth0 [10.132.0.160/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Add eth0 [10.133.0.115/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Container started 50m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Container created 50m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Container created 50m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Container started 50m Normal Killing pod/e2e-mt-db61066b-openshift-default-f9db4cc6f-b6sb9 Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Deleted pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm 50m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-f5c1ab-openshift-default Scaled up replica set e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f from 0 to 1 50m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl Stopping container istio-proxy 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 7; reason: All metrics below target 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled down replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 10 to 7 50m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Deleted pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-k9knl 50m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f Created pod: e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 50m Normal AddedLabel gateway/e2e-shared-b-w0-f5c1ab Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-f5c1ab 50m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-wj6fm Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Deleted pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-5n8bf 50m Normal AddedInterface pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Add eth0 [10.133.0.116/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/e2e-api-a-2d319a-openshift-default-64d669fc9c Created pod: e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm 50m Normal AddedLabel gateway/e2e-api-a-2d319a Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-2d319a 50m Normal ScalingReplicaSet deployment/e2e-api-a-2d319a-openshift-default Scaled up replica set e2e-api-a-2d319a-openshift-default-64d669fc9c from 0 to 1 50m Normal AddedInterface pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Add eth0 [10.133.0.117/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99 Created pod: payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused 49m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99 Error creating: pods "payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-f5c1ab: serviceaccount "payload-processing-e2e-shared-b-w0-f5c1ab" not found 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-f5c1ab Scaled up replica set payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99 from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-f5c1ab: serviceaccount "payload-processing-e2e-shared-b-w0-f5c1ab" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7 Created pod: payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-f5c1ab Scaled up replica set payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7 from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-api-a-2d319a-7dcc698947 Error creating: pods "payload-processing-e2e-api-a-2d319a-7dcc698947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2d319a: serviceaccount "payload-processing-e2e-api-a-2d319a" not found 49m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll Container created 49m Normal Started pod/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 Container started 49m Normal Created pod/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 Container created 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused 49m Normal Pulled pod/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 Add eth0 [10.133.0.119/23] from ovn-kubernetes 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused... 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-2d319a Scaled up replica set payload-processing-e2e-api-a-2d319a-7dcc698947 from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-2d319a-d95b54579 Error creating: pods "payload-pre-processing-e2e-api-a-2d319a-d95b54579-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2d319a: serviceaccount "payload-processing-e2e-api-a-2d319a" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-2d319a-d95b54579 Created pod: payload-pre-processing-e2e-api-a-2d319a-d95b54579-hzwkl 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-2d319a Scaled up replica set payload-pre-processing-e2e-api-a-2d319a-d95b54579 from 0 to 1 49m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll Add eth0 [10.133.0.118/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll Container started 49m Normal Created pod/payload-pre-processing-e2e-api-a-2d319a-d95b54579-hzwkl Container created 49m Normal Started pod/payload-pre-processing-e2e-api-a-2d319a-d95b54579-hzwkl Container started 49m Normal Pulled pod/payload-processing-e2e-api-a-2d319a-7dcc698947-479ft Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-api-a-2d319a-7dcc698947-479ft Add eth0 [10.133.0.121/23] from ovn-kubernetes 49m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-2d319a-d95b54579-hzwkl Add eth0 [10.133.0.120/23] from ovn-kubernetes 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Startup probe error: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Startup probe failed: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused 49m Normal Pulled pod/payload-pre-processing-e2e-api-a-2d319a-d95b54579-hzwkl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-api-a-2d319a-7dcc698947-479ft Container started 49m Normal Created pod/payload-processing-e2e-api-a-2d319a-7dcc698947-479ft Container created 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-2d319a-7dcc698947 Created pod: payload-processing-e2e-api-a-2d319a-7dcc698947-479ft 49m Normal AddedLabel gateway/e2e-inf-b-88eaf9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-88eaf9 49m Normal ScalingReplicaSet deployment/e2e-inf-b-88eaf9-openshift-default Scaled up replica set e2e-inf-b-88eaf9-openshift-default-6c9fd5847f from 0 to 1 49m Normal SuccessfulCreate replicaset/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f Created pod: e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd 49m Normal AddedInterface pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Add eth0 [10.132.0.161/23] from ovn-kubernetes 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-d4b21c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-d4b21c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-67adab47-openshift-default New size: 10; reason: 49m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd 49m Normal ScalingReplicaSet deployment/e2e-mt-67adab47-openshift-default Scaled up replica set e2e-mt-67adab47-openshift-default-84fb98c68f from 8 to 10 49m Normal SuccessfulCreate replicaset/e2e-mt-67adab47-openshift-default-84fb98c68f (combined from similar events): Created pod: e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l 49m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-88eaf9-56b45b6577 Error creating: pods "payload-processing-e2e-inf-b-88eaf9-56b45b6577-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-88eaf9: serviceaccount "payload-processing-e2e-inf-b-88eaf9" not found 49m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Add eth0 [10.134.0.155/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Add eth0 [10.132.0.162/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-88eaf9 Scaled up replica set payload-pre-processing-e2e-inf-b-88eaf9-697ffc695 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695 Created pod: payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695 Error creating: pods "payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-88eaf9: serviceaccount "payload-processing-e2e-inf-b-88eaf9" not found 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-88eaf9 Scaled up replica set payload-processing-e2e-inf-b-88eaf9-56b45b6577 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-88eaf9-56b45b6577 Created pod: payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p 49m Normal Created pod/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 Container created 49m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Container started 49m Normal Pulled pod/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 Add eth0 [10.133.0.123/23] from ovn-kubernetes 49m Normal Started pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Container started 49m Normal Started pod/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 Container started 49m Normal Started pod/payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p Container started 49m Normal Created pod/payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p Container created 49m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Container created 49m Normal Pulled pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p Add eth0 [10.133.0.124/23] from ovn-kubernetes 49m Normal Created pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Container created 49m Normal Pulled pod/payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Startup probe failed: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Startup probe error: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-fd1c91-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-fd1c91-openshift-default Scaled up replica set e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff from 1 to 4 49m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Created pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk 49m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Created pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm 49m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Created pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt 49m Normal Started pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt Container started 49m Normal Created pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm Container created 49m Normal AddedInterface pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Add eth0 [10.134.0.156/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt Add eth0 [10.132.0.164/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt Container created 49m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm Add eth0 [10.133.0.125/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz Add eth0 [10.133.0.126/23] from ovn-kubernetes 49m Normal Started pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm Container started 49m Normal Created pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Container created 49m Normal ScalingReplicaSet deployment/e2e-api-b-d4b21c-openshift-default Scaled up replica set e2e-api-b-d4b21c-openshift-default-5f69685d47 from 4 to 8 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-d4b21c-openshift-default New size: 8; reason: All metrics below target 49m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh 49m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Add eth0 [10.132.0.165/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz 49m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs 49m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 49m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Container started 49m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs Add eth0 [10.134.0.157/23] from ovn-kubernetes 49m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh Container started 49m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Container created 49m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz Container created 49m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh Container created 49m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh Add eth0 [10.132.0.166/23] from ovn-kubernetes 49m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Container started 49m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs Container started 49m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz Container started 49m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs Container created 49m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Readiness probe failed: Get "http://10.133.0.105:15021/healthz/ready": read tcp 10.133.0.2:56576->10.133.0.105:15021: read: connection reset by peer 49m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Readiness probe error: Get "http://10.133.0.105:15021/healthz/ready": read tcp 10.133.0.2:56576->10.133.0.105:15021: read: connection reset by peer... 49m Normal ScalingReplicaSet deployment/e2e-api-a-e73256-openshift-default Scaled up replica set e2e-api-a-e73256-openshift-default-dfc9897d9 from 1 to 4 49m Normal SuccessfulCreate replicaset/e2e-api-a-e73256-openshift-default-dfc9897d9 Created pod: e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq 49m Normal SuccessfulCreate replicaset/e2e-api-a-e73256-openshift-default-dfc9897d9 Created pod: e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv 49m Normal Pulled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-api-a-e73256-openshift-default-dfc9897d9 Created pod: e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp 49m Normal Created pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Container created 49m Normal Started pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Container started 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-e73256-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulDelete replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Deleted pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 49m Normal SuccessfulDelete replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Deleted pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf 49m Normal SuccessfulDelete replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Deleted pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled down replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 10 to 7 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 6; reason: All metrics below target 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 6; reason: All metrics below target 49m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled down replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 7 to 6 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 7; reason: All metrics below target 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled down replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 7 to 6 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-f5c1ab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal SuccessfulDelete replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Deleted pod: e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr 49m Normal SuccessfulDelete replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Deleted pod: e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx 49m Normal SuccessfulDelete replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Deleted pod: e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h 49m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Stopping container istio-proxy 49m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-qv422 Stopping container istio-proxy 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-f5c1ab-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled down replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 7 to 6 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 6; reason: All metrics below target 49m Normal Pulled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp Add eth0 [10.134.0.159/23] from ovn-kubernetes 49m Normal Created pod/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv Container created 49m Normal Pulled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv Add eth0 [10.134.0.158/23] from ovn-kubernetes 49m Normal Started pod/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv Container started 49m Normal Started pod/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp Container started 49m Normal Created pod/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp Container created 49m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Stopping container istio-proxy 49m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-kj2wx Stopping container istio-proxy 49m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-2zkcf Stopping container istio-proxy 49m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-85w4h Stopping container istio-proxy 49m Normal AddedInterface pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Add eth0 [10.132.0.167/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Readiness probe error: Get "http://10.132.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Stopping container istio-proxy 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Stopping container istio-proxy 49m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz Startup probe error: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz Startup probe failed: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Stopping container istio-proxy 49m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm Startup probe error: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Readiness probe failed: Get "http://10.132.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Stopping container istio-proxy 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Stopping container istio-proxy 49m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm Startup probe failed: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Stopping container istio-proxy 49m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Startup probe error: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Startup probe failed: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused 49m Normal Created pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Container created 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Startup probe error: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Startup probe failed: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh Startup probe error: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt Startup probe error: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh Startup probe failed: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt Startup probe failed: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-vx2qt Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w8vmw Stopping container istio-proxy 49m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Startup probe error: Get "http://10.132.0.165:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Readiness probe failed: HTTP probe failed with statuscode: 503 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Stopping container istio-proxy 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Stopping container istio-proxy 49m Normal Killing pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Stopping container istio-proxy 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-49sbg Readiness probe error: HTTP probe failed with statuscode: 503... 49m Normal Started pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Container started 49m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Startup probe failed: Get "http://10.132.0.165:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-2bcjd Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Readiness probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-ffmnr Readiness probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tpv89 Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-hmf8l Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-sw272 Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-5t4q7 Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning Unhealthy pod/e2e-mt-67adab47-openshift-default-84fb98c68f-tqljf Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning ProbeError pod/e2e-mt-67adab47-openshift-default-84fb98c68f-w2tzg Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Readiness probe error: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-cnj52 Readiness probe failed: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-fd1c91-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-fd1c91-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Started pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Container started 49m Normal Created pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Container created 49m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp Startup probe failed: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv Startup probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 49m Normal SuccessfulCreate replicaset/e2e-api-b-d4b21c-openshift-default-5f69685d47 Created pod: e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn 49m Normal Pulled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-d4b21c-openshift-default New size: 9; reason: All metrics below target 49m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv Startup probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp Startup probe error: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused... 49m Normal ScalingReplicaSet deployment/e2e-api-b-d4b21c-openshift-default Scaled up replica set e2e-api-b-d4b21c-openshift-default-5f69685d47 from 8 to 9 49m Normal Pulled pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-fh4rc Stopping container istio-proxy 49m Normal Started pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Container started 49m Normal Created pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Container created 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-xcrcs Stopping container istio-proxy 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ls6dw Stopping container istio-proxy 49m Normal AddedInterface pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Add eth0 [10.134.0.160/23] from ovn-kubernetes 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-79qgz Stopping container istio-proxy 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-wqr6z Stopping container istio-proxy 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e73256-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Startup probe error: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused... 49m Normal SuccessfulCreate replicaset/e2e-inf-a-a4e312-openshift-default-65c69cb84c Created pod: e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-ht5g8 Stopping container istio-proxy 49m Warning FailedToRetrieveImagePullSecret pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Unable to retrieve some image pull secrets (e2e-api-b-d4b21c-openshift-default-dockercfg-k8zbt); attempting to pull the image may not succeed. 49m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Startup probe error: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Startup probe failed: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e73256-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Startup probe failed: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused 49m Normal AddedLabel gateway/e2e-inf-a-a4e312 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-a4e312 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Stopping container istio-proxy 49m Normal ScalingReplicaSet deployment/e2e-inf-a-a4e312-openshift-default Scaled up replica set e2e-inf-a-a4e312-openshift-default-65c69cb84c from 0 to 1 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-lrglh Stopping container istio-proxy 49m Normal Killing pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-cg6tn Stopping container istio-proxy 49m Normal AddedInterface pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Add eth0 [10.132.0.168/23] from ovn-kubernetes 49m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Startup probe error: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Startup probe failed: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning Unhealthy pod/e2e-api-b-d4b21c-openshift-default-5f69685d47-5rfdg Readiness probe failed: HTTP probe failed with statuscode: 503 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-a4e312 Scaled up replica set payload-processing-e2e-inf-a-a4e312-7b78fb8665 from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-a4e312-7b78fb8665 Error creating: pods "payload-processing-e2e-inf-a-a4e312-7b78fb8665-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a4e312: serviceaccount "payload-processing-e2e-inf-a-a4e312" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-a4e312-7b78fb8665 Created pod: payload-processing-e2e-inf-a-a4e312-7b78fb8665-b9pn6 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-a4e312-74585c8467 Error creating: pods "payload-pre-processing-e2e-inf-a-a4e312-74585c8467-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a4e312: serviceaccount "payload-processing-e2e-inf-a-a4e312" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-a4e312-74585c8467 Created pod: payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-a4e312 Scaled up replica set payload-pre-processing-e2e-inf-a-a4e312-74585c8467 from 0 to 1 49m Normal Started pod/payload-processing-e2e-inf-a-a4e312-7b78fb8665-b9pn6 Container started 49m Normal Created pod/payload-processing-e2e-inf-a-a4e312-7b78fb8665-b9pn6 Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j Add eth0 [10.132.0.170/23] from ovn-kubernetes 49m Normal AddedInterface pod/payload-processing-e2e-inf-a-a4e312-7b78fb8665-b9pn6 Add eth0 [10.133.0.129/23] from ovn-kubernetes 49m Normal Created pod/payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j Container created 49m Normal Pulled pod/payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Pulled pod/payload-processing-e2e-inf-a-a4e312-7b78fb8665-b9pn6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j Container started 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-fd1c91-openshift-default New size: 6; reason: 49m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Created pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv 49m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Created pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 49m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-fd1c91-openshift-default Scaled up replica set e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff from 4 to 6 49m Normal Created pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Container created 49m Normal AddedInterface pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 Add eth0 [10.134.0.161/23] from ovn-kubernetes 49m Normal Killing pod/payload-pre-processing-e2e-api-a-e73256-b9d6b9887-62lj9 Stopping container payload-pre-processing 49m Normal Started pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Container started 49m Normal Pulled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 Container started 49m Normal Created pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 Container created 49m Normal AddedInterface pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Add eth0 [10.132.0.171/23] from ovn-kubernetes 49m Normal Killing pod/payload-processing-e2e-api-a-e73256-57b5c8849f-n9qzl Stopping container payload-processing 49m Normal Pulled pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-inf-a-278c04-openshift-default-9cb8c9ff5 Created pod: e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj 49m Normal AddedLabel gateway/e2e-inf-a-278c04 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-278c04 49m Normal ScalingReplicaSet deployment/e2e-inf-a-278c04-openshift-default Scaled up replica set e2e-inf-a-278c04-openshift-default-9cb8c9ff5 from 0 to 1 49m Normal AddedInterface pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj Add eth0 [10.132.0.172/23] from ovn-kubernetes 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e73256-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e73256-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled down replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 6 to 5 49m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr Stopping container istio-proxy 49m Normal SuccessfulDelete replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Deleted pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-s8lwr 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 5; reason: All metrics below target 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-278c04 Scaled up replica set payload-processing-e2e-inf-a-278c04-6969d7fcb9 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc Created pod: payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg 49m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 49m Normal AddedLabel gateway/e2e-api-b-a813e2 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-a813e2 49m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 49m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-278c04-6969d7fcb9 Error creating: pods "payload-processing-e2e-inf-a-278c04-6969d7fcb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-278c04: serviceaccount "payload-processing-e2e-inf-a-278c04" not found 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc Error creating: pods "payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-278c04: serviceaccount "payload-processing-e2e-inf-a-278c04" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-278c04-6969d7fcb9 Created pod: payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn 49m Normal ScalingReplicaSet deployment/e2e-api-b-a813e2-openshift-default Scaled up replica set e2e-api-b-a813e2-openshift-default-756f6db67 from 0 to 1 49m Normal Pulled pod/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg Add eth0 [10.133.0.130/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-api-b-a813e2-openshift-default-756f6db67 Created pod: e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-278c04 Scaled up replica set payload-pre-processing-e2e-inf-a-278c04-86645f6fbc from 0 to 1 48m Normal AddedInterface pod/payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn Add eth0 [10.133.0.131/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused 48m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused... 48m Normal Started pod/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg Container started 48m Normal AddedInterface pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Add eth0 [10.133.0.132/23] from ovn-kubernetes 48m Normal Started pod/payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn Container started 48m Normal Created pod/payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn Container created 48m Normal Created pod/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg Container created 48m Warning FailedCreate replicaset/payload-processing-e2e-api-b-a813e2-dbc7795cc Error creating: pods "payload-processing-e2e-api-b-a813e2-dbc7795cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-a813e2: serviceaccount "payload-processing-e2e-api-b-a813e2" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-a813e2 Scaled up replica set payload-pre-processing-e2e-api-b-a813e2-747f99fd79 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-a813e2 Scaled up replica set payload-processing-e2e-api-b-a813e2-dbc7795cc from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-a813e2-747f99fd79 Error creating: pods "payload-pre-processing-e2e-api-b-a813e2-747f99fd79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-a813e2: serviceaccount "payload-processing-e2e-api-b-a813e2" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-a813e2-dbc7795cc Created pod: payload-processing-e2e-api-b-a813e2-dbc7795cc-vqxqm 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-a813e2-747f99fd79 Created pod: payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw 48m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Readiness probe error: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Readiness probe error: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal AddedInterface pod/payload-processing-e2e-api-b-a813e2-dbc7795cc-vqxqm Add eth0 [10.133.0.134/23] from ovn-kubernetes 48m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw Add eth0 [10.133.0.133/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-api-a-e73256-openshift-default-dfc9897d9 Created pod: e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p 48m Normal SuccessfulCreate replicaset/e2e-api-a-e73256-openshift-default-dfc9897d9 Created pod: e2e-api-a-e73256-openshift-default-dfc9897d9-46875 48m Normal Started pod/payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw Container started 48m Normal SuccessfulCreate replicaset/e2e-api-a-e73256-openshift-default-dfc9897d9 Created pod: e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 48m Normal ScalingReplicaSet deployment/e2e-api-a-e73256-openshift-default Scaled up replica set e2e-api-a-e73256-openshift-default-dfc9897d9 from 4 to 7 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-e73256-openshift-default New size: 7; reason: 48m Normal Pulled pod/payload-processing-e2e-api-b-a813e2-dbc7795cc-vqxqm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw Container created 48m Normal AddedInterface pod/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 Add eth0 [10.133.0.136/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p Container created 48m Normal Started pod/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p Container started 48m Normal AddedInterface pod/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p Add eth0 [10.132.0.173/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-api-b-a813e2-dbc7795cc-vqxqm Container created 48m Normal Started pod/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 Container started 48m Normal Created pod/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 Container created 48m Normal Started pod/payload-processing-e2e-api-b-a813e2-dbc7795cc-vqxqm Container started 48m Normal Pulled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 Container started 48m Normal Created pod/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 Container created 48m Normal Pulled pod/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 Add eth0 [10.133.0.137/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Readiness probe error: Get "http://10.133.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Readiness probe failed: Get "http://10.133.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Readiness probe error: Get "http://10.133.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Readiness probe failed: Get "http://10.133.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 Startup probe error: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p Startup probe failed: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 48m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p Startup probe error: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 Startup probe failed: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused 48m Normal Pulled pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Container created 48m Normal Started pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Container started 48m Normal Started pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Container started 48m Normal Created pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Container created 48m Normal Pulled pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Container started 48m Normal Pulled pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Container created 48m Normal SuccessfulCreate replicaset/e2e-inf-a-b1fae7-openshift-default-697b4b59f Created pod: e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq 48m Normal AddedLabel gateway/e2e-inf-a-b1fae7 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-b1fae7 48m Normal ScalingReplicaSet deployment/e2e-inf-a-b1fae7-openshift-default Scaled up replica set e2e-inf-a-b1fae7-openshift-default-697b4b59f from 0 to 1 48m Normal AddedInterface pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Add eth0 [10.132.0.177/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 48m Warning ProbeError pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-2d319a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-2d319a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-a813e2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Startup probe error: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Startup probe failed: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-a813e2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5 Created pod: payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-x9g5w 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5 Error creating: pods "payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b1fae7: serviceaccount "payload-processing-e2e-inf-a-b1fae7" not found 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c Error creating: pods "payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b1fae7: serviceaccount "payload-processing-e2e-inf-a-b1fae7" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c Created pod: payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-b1fae7 Scaled up replica set payload-processing-e2e-inf-a-b1fae7-6c4988f4d5 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-b1fae7 Scaled up replica set payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c from 0 to 1 48m Normal Created pod/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-x9g5w Container created 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz Add eth0 [10.134.0.163/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz Container started 48m Normal Created pod/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz Container created 48m Normal AddedInterface pod/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-x9g5w Add eth0 [10.134.0.164/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Pulled pod/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-x9g5w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-x9g5w Container started 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-88eaf9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-88eaf9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal SuccessfulCreate replicaset/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc Created pod: e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp 48m Normal AddedLabel gateway/e2e-inf-b-d45f6c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-d45f6c 48m Normal ScalingReplicaSet deployment/e2e-inf-b-d45f6c-openshift-default Scaled up replica set e2e-inf-b-d45f6c-openshift-default-7b5486ffcc from 0 to 1 48m Normal AddedInterface pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Add eth0 [10.133.0.139/23] from ovn-kubernetes 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-f5c1ab-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-f5c1ab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-d45f6c Scaled up replica set payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9 Error creating: pods "payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d45f6c: serviceaccount "payload-processing-e2e-inf-b-d45f6c" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9 Created pod: payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-d45f6c Scaled up replica set payload-processing-e2e-inf-b-d45f6c-59d8bb7487 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-d45f6c-59d8bb7487 Created pod: payload-processing-e2e-inf-b-d45f6c-59d8bb7487-tcsb7 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-d45f6c-59d8bb7487 Error creating: pods "payload-processing-e2e-inf-b-d45f6c-59d8bb7487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d45f6c: serviceaccount "payload-processing-e2e-inf-b-d45f6c" not found 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg Add eth0 [10.133.0.140/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-inf-b-d45f6c-59d8bb7487-tcsb7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-inf-b-d45f6c-59d8bb7487-tcsb7 Container created 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-inf-b-d45f6c-59d8bb7487-tcsb7 Container started 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-d45f6c-59d8bb7487-tcsb7 Add eth0 [10.134.0.165/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg Container created 48m Normal Started pod/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg Container started 48m Normal AddedLabel gateway/e2e-inf-b-9efeae Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-9efeae 48m Normal ScalingReplicaSet deployment/e2e-inf-b-9efeae-openshift-default Scaled up replica set e2e-inf-b-9efeae-openshift-default-6df74d98cf from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-b-9efeae-openshift-default-6df74d98cf Created pod: e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z 48m Normal AddedInterface pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Add eth0 [10.133.0.141/23] from ovn-kubernetes 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-88eaf9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-88eaf9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Readiness probe error: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Killing pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Stopping container istio-proxy 48m Normal Killing pod/e2e-api-a-e73256-openshift-default-dfc9897d9-46875 Stopping container istio-proxy 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal Killing pod/e2e-api-a-e73256-openshift-default-dfc9897d9-bbz5p Stopping container istio-proxy 48m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Readiness probe failed: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal Killing pod/e2e-api-a-e73256-openshift-default-dfc9897d9-xf6w7 Stopping container istio-proxy 48m Normal Killing pod/e2e-api-a-e73256-openshift-default-dfc9897d9-gwppv Stopping container istio-proxy 48m Normal Killing pod/e2e-api-a-e73256-openshift-default-dfc9897d9-wcgmp Stopping container istio-proxy 48m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Readiness probe error: Get "http://10.133.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Killing pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Stopping container istio-proxy 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Readiness probe failed: Get "http://10.133.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal ScalingReplicaSet deployment/e2e-inf-a-1e8cf0-openshift-default Scaled up replica set e2e-inf-a-1e8cf0-openshift-default-68f469b489 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-a-1e8cf0-openshift-default-68f469b489 Created pod: e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb 48m Normal AddedLabel gateway/e2e-inf-a-1e8cf0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-1e8cf0 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5 Created pod: payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-9efeae-6d89d89df7 Error creating: pods "payload-processing-e2e-inf-b-9efeae-6d89d89df7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-9efeae: serviceaccount "payload-processing-e2e-inf-b-9efeae" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-9efeae-6d89d89df7 Created pod: payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-9efeae Scaled up replica set payload-processing-e2e-inf-b-9efeae-6d89d89df7 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5 Error creating: pods "payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-9efeae: serviceaccount "payload-processing-e2e-inf-b-9efeae" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-9efeae Scaled up replica set payload-pre-processing-e2e-inf-b-9efeae-699f495dc5 from 0 to 1 48m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc Add eth0 [10.134.0.167/23] from ovn-kubernetes 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l Add eth0 [10.134.0.166/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-dwnsq Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-2d319a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal Pulled pod/payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc Container created 48m Normal Started pod/payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc Container started 48m Normal Started pod/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l Container started 48m Normal Created pod/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l Container created 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-2d319a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning ProbeError pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Readiness probe error: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-api-a-e73256-openshift-default-dfc9897d9-mg2hk Readiness probe failed: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused 47m Normal AddedInterface pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Add eth0 [10.132.0.179/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Readiness probe failed: Get "http://10.133.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Readiness probe error: Get "http://10.133.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal Started pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Container started 47m Normal Pulled pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Container created 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f Error creating: pods "payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-1e8cf0: serviceaccount "payload-processing-e2e-inf-a-1e8cf0" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-1e8cf0 Scaled up replica set payload-processing-e2e-inf-a-1e8cf0-56bb54564c from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-1e8cf0-56bb54564c Error creating: pods "payload-processing-e2e-inf-a-1e8cf0-56bb54564c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-1e8cf0: serviceaccount "payload-processing-e2e-inf-a-1e8cf0" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-1e8cf0 Scaled up replica set payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f Created pod: payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-1e8cf0-56bb54564c Created pod: payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-278c04-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 47m Normal Created pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v Container created 47m Normal Pulled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal ScalingReplicaSet deployment/e2e-inf-a-278c04-openshift-default Scaled up replica set e2e-inf-a-278c04-openshift-default-9cb8c9ff5 from 1 to 4 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb Add eth0 [10.132.0.181/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb Container created 47m Normal Started pod/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb Container started 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 Add eth0 [10.133.0.142/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v Add eth0 [10.134.0.169/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/e2e-inf-a-278c04-openshift-default-9cb8c9ff5 Created pod: e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch 47m Normal SuccessfulCreate replicaset/e2e-inf-a-278c04-openshift-default-9cb8c9ff5 Created pod: e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v 47m Normal SuccessfulCreate replicaset/e2e-inf-a-278c04-openshift-default-9cb8c9ff5 Created pod: e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n 47m Normal Started pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Container started 47m Normal Created pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Container created 47m Normal Pulled pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 Container started 47m Normal Created pod/payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 Container created 47m Normal Pulled pod/payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch Add eth0 [10.132.0.182/23] from ovn-kubernetes 47m Normal Started pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v Container started 47m Normal AddedInterface pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n Add eth0 [10.133.0.143/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n Container created 47m Normal Started pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n Container started 47m Normal Started pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch Container started 47m Normal Created pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch Container created 47m Normal Pulled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj Container created 47m Normal Started pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj Container started 47m Warning Unhealthy pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Startup probe failed: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Startup probe error: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused... 47m Normal Killing pod/payload-pre-processing-e2e-api-b-1c0c1d-65fc757fd7-c79ts Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-api-b-1c0c1d-758fdd696f-ff9zq Stopping container payload-processing 47m Warning ProbeError pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v Startup probe error: Get "http://10.134.0.169:15021/healthz/ready": dial tcp 10.134.0.169:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v Startup probe failed: Get "http://10.134.0.169:15021/healthz/ready": dial tcp 10.134.0.169:15021: connect: connection refused 47m Warning ProbeError pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch Startup probe error: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch Startup probe failed: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Startup probe failed: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused 47m Warning ProbeError pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Startup probe error: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused... 47m Normal Killing pod/payload-pre-processing-e2e-api-a-2d319a-d95b54579-hzwkl Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-api-a-2d319a-7dcc698947-479ft Stopping container payload-processing 47m Warning ProbeError pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj Startup probe error: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj Startup probe failed: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-278c04-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-278c04-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Started pod/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 Container started 47m Normal Pulled pod/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 Container created 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-03dfee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-03dfee-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal SuccessfulCreate replicaset/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc Created pod: e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 47m Normal AddedLabel gateway/e2e-inf-b-f8c3f7 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-f8c3f7 47m Normal ScalingReplicaSet deployment/e2e-inf-b-f8c3f7-openshift-default Scaled up replica set e2e-inf-b-f8c3f7-openshift-default-78b8d845bc from 0 to 1 47m Warning Unhealthy pod/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 Startup probe failed: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused 47m Normal AddedInterface pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Add eth0 [10.133.0.144/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 Startup probe error: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused... 47m Normal SuccessfulCreate replicaset/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f Created pod: e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb 47m Normal ScalingReplicaSet deployment/e2e-inf-b-88eaf9-openshift-default Scaled up replica set e2e-inf-b-88eaf9-openshift-default-6c9fd5847f from 1 to 2 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-88eaf9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c Created pod: payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z 47m Normal Started pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Container started 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c Error creating: pods "payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f8c3f7: serviceaccount "payload-processing-e2e-inf-b-f8c3f7" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-f8c3f7 Scaled up replica set payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-f8c3f7 Scaled up replica set payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7 Created pod: payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7 Error creating: pods "payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f8c3f7: serviceaccount "payload-processing-e2e-inf-b-f8c3f7" not found 47m Normal AddedInterface pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Add eth0 [10.133.0.145/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Container created 47m Normal Created pod/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm Container created 47m Normal Pulled pod/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z Container created 47m Normal Pulled pod/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm Add eth0 [10.133.0.146/23] from ovn-kubernetes 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z Add eth0 [10.134.0.171/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm Container started 47m Normal Started pod/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z Container started 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-278c04-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-278c04-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-1c0c1d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-1c0c1d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning ProbeError pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Startup probe error: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Startup probe failed: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-03dfee-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-03dfee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-a813e2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-a813e2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning ProbeError pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Readiness probe error: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-1c0c1d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-1c0c1d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning Unhealthy pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Readiness probe failed: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal AddedLabel gateway/e2e-inf-b-8dcfa8 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-8dcfa8 47m Warning ProbeError pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Readiness probe error: Get "http://10.132.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Readiness probe failed: Get "http://10.132.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Killing pod/e2e-api-b-1c0c1d-openshift-default-857c68594c-pzw96 Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-inf-b-8dcfa8-openshift-default Scaled up replica set e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf Created pod: e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz 47m Normal AddedLabel gateway/e2e-inf-a-785d26 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-785d26 47m Normal SuccessfulCreate replicaset/e2e-inf-a-785d26-openshift-default-74465b794c Created pod: e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb 47m Normal ScalingReplicaSet deployment/e2e-inf-a-785d26-openshift-default Scaled up replica set e2e-inf-a-785d26-openshift-default-74465b794c from 0 to 1 46m Normal AddedInterface pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Add eth0 [10.132.0.186/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Add eth0 [10.132.0.185/23] from ovn-kubernetes 46m Normal Started pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Container started 46m Normal Created pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Container created 46m Normal Pulled pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Container started 46m Normal Pulled pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Container created 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-8dcfa8-86d859f49f Created pod: payload-processing-e2e-inf-b-8dcfa8-86d859f49f-j5f54 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-8dcfa8 Scaled up replica set payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5 Created pod: payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5 Error creating: pods "payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8dcfa8: serviceaccount "payload-processing-e2e-inf-b-8dcfa8" not found 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-8dcfa8-86d859f49f Error creating: pods "payload-processing-e2e-inf-b-8dcfa8-86d859f49f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8dcfa8: serviceaccount "payload-processing-e2e-inf-b-8dcfa8" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-8dcfa8 Scaled up replica set payload-processing-e2e-inf-b-8dcfa8-86d859f49f from 0 to 1 46m Normal AddedInterface pod/payload-processing-e2e-inf-b-8dcfa8-86d859f49f-j5f54 Add eth0 [10.133.0.149/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv Add eth0 [10.133.0.148/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-785d26-6467945f49 Created pod: payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-785d26 Scaled up replica set payload-processing-e2e-inf-a-785d26-6467945f49 from 0 to 1 46m Normal Created pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Container created 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-785d26-6467945f49 Error creating: pods "payload-processing-e2e-inf-a-785d26-6467945f49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-785d26: serviceaccount "payload-processing-e2e-inf-a-785d26" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f Error creating: pods "payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-785d26: serviceaccount "payload-processing-e2e-inf-a-785d26" not found 46m Normal Started pod/payload-processing-e2e-inf-b-8dcfa8-86d859f49f-j5f54 Container started 46m Normal Created pod/payload-processing-e2e-inf-b-8dcfa8-86d859f49f-j5f54 Container created 46m Normal Pulled pod/payload-processing-e2e-inf-b-8dcfa8-86d859f49f-j5f54 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f Created pod: payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-785d26 Scaled up replica set payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f from 0 to 1 46m Normal Started pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Container started 46m Normal Started pod/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv Container started 46m Normal Created pod/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv Container created 46m Normal Pulled pod/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Pulled pod/payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc Add eth0 [10.133.0.151/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 Add eth0 [10.133.0.152/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc Container created 46m Normal Started pod/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc Container started 46m Normal Created pod/payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 Container created 46m Normal Started pod/payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 Container started 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b1fae7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b1fae7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning ProbeError pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Startup probe failed: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Startup probe error: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 46m Normal Killing pod/e2e-api-a-2d319a-openshift-default-64d669fc9c-wjjhm Stopping container istio-proxy 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b1fae7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b1fae7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 6; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled down replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 7 to 6 46m Normal Killing pod/payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw Stopping container payload-pre-processing 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Deleted pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-cbxtq 46m Normal Killing pod/payload-processing-e2e-api-b-a813e2-dbc7795cc-vqxqm Stopping container payload-processing 46m Warning Unhealthy pod/payload-pre-processing-e2e-api-b-a813e2-747f99fd79-prchw Readiness probe failed: timeout: failed to connect service "10.133.0.133:50052" within 1s: context deadline exceeded 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d45f6c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d45f6c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-9efeae-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-9efeae-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Created pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Container created 46m Normal Pulled pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Container started 45m Warning Unhealthy pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Readiness probe failed: Get "http://10.132.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Readiness probe error: Get "http://10.132.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal AddedLabel gateway/e2e-inf-b-d8c398 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-d8c398 45m Normal ScalingReplicaSet deployment/e2e-inf-b-d8c398-openshift-default Scaled up replica set e2e-inf-b-d8c398-openshift-default-65cfb69fd7 from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-inf-b-d8c398-openshift-default-65cfb69fd7 Created pod: e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f8c3f7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f8c3f7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal AddedInterface pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Add eth0 [10.132.0.189/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Container created 45m Normal Started pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Container started 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-d8c398 Scaled up replica set payload-processing-e2e-inf-b-d8c398-546ff884f from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-d8c398-546ff884f Error creating: pods "payload-processing-e2e-inf-b-d8c398-546ff884f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d8c398: serviceaccount "payload-processing-e2e-inf-b-d8c398" not found 45m Warning ProbeError pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-d8c398 Scaled up replica set payload-pre-processing-e2e-inf-b-d8c398-69fd677f from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-d8c398-69fd677f Error creating: pods "payload-pre-processing-e2e-inf-b-d8c398-69fd677f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d8c398: serviceaccount "payload-processing-e2e-inf-b-d8c398" not found 45m Warning Unhealthy pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-d8c398-69fd677f Created pod: payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc Add eth0 [10.133.0.159/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-d8c398-546ff884f Created pod: payload-processing-e2e-inf-b-d8c398-546ff884f-pr754 45m Normal Created pod/payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc Container created 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-d45f6c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Normal ScalingReplicaSet deployment/e2e-inf-b-d45f6c-openshift-default Scaled up replica set e2e-inf-b-d45f6c-openshift-default-7b5486ffcc from 1 to 2 45m Normal Pulled pod/payload-processing-e2e-inf-b-d8c398-546ff884f-pr754 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-inf-b-d8c398-546ff884f-pr754 Container started 45m Normal SuccessfulCreate replicaset/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc Created pod: e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq 45m Normal Created pod/payload-processing-e2e-inf-b-d8c398-546ff884f-pr754 Container created 45m Normal Started pod/payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc Container started 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-d8c398-546ff884f-pr754 Add eth0 [10.134.0.173/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-inf-a-278c04-openshift-default-9cb8c9ff5 Created pod: e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v 45m Normal Started pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Container started 45m Normal Created pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Container created 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-278c04-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/e2e-inf-a-278c04-openshift-default-9cb8c9ff5 Created pod: e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs 45m Warning Unhealthy pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Startup probe failed: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused 45m Normal ScalingReplicaSet deployment/e2e-inf-a-278c04-openshift-default Scaled up replica set e2e-inf-a-278c04-openshift-default-9cb8c9ff5 from 4 to 6 45m Normal Pulled pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Add eth0 [10.133.0.174/23] from ovn-kubernetes 45m Warning ProbeError pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Startup probe error: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a4e312-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a4e312-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal ScalingReplicaSet deployment/e2e-inf-a-03dfee-openshift-default Scaled up replica set e2e-inf-a-03dfee-openshift-default-5ffbd99665 from 1 to 2 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-03dfee-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Warning Unhealthy pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Startup probe failed: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused 45m Warning ProbeError pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Startup probe error: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused... 45m Normal SuccessfulCreate replicaset/e2e-inf-a-03dfee-openshift-default-5ffbd99665 Created pod: e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf 45m Normal Pulled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Container created 45m Normal Started pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Container started 45m Normal AddedInterface pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Add eth0 [10.134.0.183/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-inf-b-9efeae-openshift-default-6df74d98cf Created pod: e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-9efeae-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Normal ScalingReplicaSet deployment/e2e-inf-b-9efeae-openshift-default Scaled up replica set e2e-inf-b-9efeae-openshift-default-6df74d98cf from 1 to 2 45m Warning ProbeError pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Startup probe error: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Startup probe failed: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a4e312-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a4e312-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal AddedInterface pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 Add eth0 [10.134.0.185/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Killing pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Stopping container istio-proxy 45m Normal Created pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 Container created 45m Normal Started pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 Container started 45m Normal AddedLabel gateway/e2e-inf-a-754e0a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-754e0a 45m Normal ScalingReplicaSet deployment/e2e-inf-a-754e0a-openshift-default Scaled up replica set e2e-inf-a-754e0a-openshift-default-7c7cdd7487 from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-inf-a-754e0a-openshift-default-7c7cdd7487 Created pod: e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 45m Warning ProbeError pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-api-b-a813e2-openshift-default-756f6db67-mzvc9 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f8c3f7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d8c398-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d8c398-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f8c3f7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning Unhealthy pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 Startup probe failed: Get "http://10.134.0.185:15021/healthz/ready": dial tcp 10.134.0.185:15021: connect: connection refused 45m Warning ProbeError pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 Startup probe error: Get "http://10.134.0.185:15021/healthz/ready": dial tcp 10.134.0.185:15021: connect: connection refused... 45m Normal AddedInterface pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Add eth0 [10.133.0.176/23] from ovn-kubernetes 45m Normal AddedInterface pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v Add eth0 [10.133.0.177/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v Container created 45m Normal Started pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v Container started 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-754e0a-7858857447 Error creating: pods "payload-pre-processing-e2e-inf-a-754e0a-7858857447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-754e0a: serviceaccount "payload-processing-e2e-inf-a-754e0a" not found 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-754e0a-85f75ff745 Error creating: pods "payload-processing-e2e-inf-a-754e0a-85f75ff745-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-754e0a: serviceaccount "payload-processing-e2e-inf-a-754e0a" not found 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-754e0a-85f75ff745 Created pod: payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-754e0a Scaled up replica set payload-processing-e2e-inf-a-754e0a-85f75ff745 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-754e0a Scaled up replica set payload-pre-processing-e2e-inf-a-754e0a-7858857447 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-754e0a-7858857447 Created pod: payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb 45m Normal Pulled pod/payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 Add eth0 [10.134.0.187/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb Add eth0 [10.134.0.186/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb Container started 45m Normal Created pod/payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 Container created 45m Normal Created pod/payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb Container created 45m Normal Started pod/payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 Container started 45m Normal Pulled pod/payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning ProbeError pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v Startup probe error: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v Startup probe failed: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Readiness probe failed: Get "http://10.132.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Readiness probe error: Get "http://10.132.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal ScalingReplicaSet deployment/e2e-inf-a-b1fae7-openshift-default Scaled up replica set e2e-inf-a-b1fae7-openshift-default-697b4b59f from 1 to 2 45m Normal SuccessfulCreate replicaset/e2e-inf-a-b1fae7-openshift-default-697b4b59f Created pod: e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg 45m Normal Started pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Container started 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-b1fae7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Normal Created pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Container created 45m Normal Pulled pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Container created 45m Normal Pulled pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Container started 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 6; reason: All metrics below target 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Deleted pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj 45m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled down replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 7 to 6 45m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Stopping container istio-proxy 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8dcfa8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8dcfa8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning Unhealthy pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 44m Warning ProbeError pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Startup probe error: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Startup probe failed: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused 44m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-pwjxj Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal Pulled pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf Add eth0 [10.134.0.188/23] from ovn-kubernetes 44m Normal Created pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf Container created 44m Normal Started pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf Container started 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-1e8cf0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-1e8cf0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning Unhealthy pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf Startup probe failed: Get "http://10.134.0.188:15021/healthz/ready": dial tcp 10.134.0.188:15021: connect: connection refused 44m Warning ProbeError pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf Startup probe error: Get "http://10.134.0.188:15021/healthz/ready": dial tcp 10.134.0.188:15021: connect: connection refused... 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-754e0a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-754e0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal ScalingReplicaSet deployment/e2e-inf-b-f8c3f7-openshift-default Scaled up replica set e2e-inf-b-f8c3f7-openshift-default-78b8d845bc from 1 to 2 44m Normal SuccessfulCreate replicaset/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc Created pod: e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-f8c3f7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-1e8cf0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-1e8cf0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 4; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 6; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled down replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 7 to 6 44m Normal SuccessfulDelete replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Deleted pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-r9b7w 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Deleted pod: e2e-worker-w0-23843f-openshift-default-54cf54885-wv98x 44m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default Scaled down replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 5 to 4 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8dcfa8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8dcfa8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Started pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg Container started 44m Normal Pulled pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Pulled pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg Container created 44m Normal AddedInterface pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg Add eth0 [10.134.0.190/23] from ovn-kubernetes 44m Normal Created pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Container created 44m Normal Started pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Container started 44m Warning Unhealthy pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg Startup probe failed: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused 44m Warning ProbeError pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg Startup probe error: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-8dcfa8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 44m Normal SuccessfulCreate replicaset/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf Created pod: e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk 44m Normal ScalingReplicaSet deployment/e2e-inf-b-8dcfa8-openshift-default Scaled up replica set e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf from 1 to 2 43m Normal AddedLabel gateway/e2e-inf-b-7c892a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-7c892a 43m Normal ScalingReplicaSet deployment/e2e-inf-b-7c892a-openshift-default Scaled up replica set e2e-inf-b-7c892a-openshift-default-c8946d869 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-inf-b-7c892a-openshift-default-c8946d869 Created pod: e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 43m Normal Created pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Container created 43m Normal Started pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Container started 43m Normal Pulled pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal ScalingReplicaSet deployment/e2e-inf-a-754e0a-openshift-default Scaled up replica set e2e-inf-a-754e0a-openshift-default-7c7cdd7487 from 1 to 4 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-754e0a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/e2e-inf-a-754e0a-openshift-default-7c7cdd7487 Created pod: e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 43m Normal SuccessfulCreate replicaset/e2e-inf-a-754e0a-openshift-default-7c7cdd7487 Created pod: e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g 43m Normal SuccessfulCreate replicaset/e2e-inf-a-754e0a-openshift-default-7c7cdd7487 Created pod: e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r 43m Normal Started pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Container started 43m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Readiness probe error: Get "http://10.133.0.176:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Created pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Container created 43m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Readiness probe failed: Get "http://10.133.0.176:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Pulled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d8c398-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d8c398-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Startup probe failed: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Startup probe error: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Startup probe failed: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused 43m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Deleted pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-fd1c91-openshift-default New size: 5; reason: All metrics below target 43m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Startup probe error: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused... 43m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-fd1c91-openshift-default Scaled down replica set e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff from 6 to 5 43m Normal Killing pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-9jz5c Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-inf-a-b1fae7-openshift-default Scaled up replica set e2e-inf-a-b1fae7-openshift-default-697b4b59f from 2 to 3 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-b1fae7-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/e2e-inf-a-b1fae7-openshift-default-697b4b59f Created pod: e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-a4e312-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/e2e-inf-a-a4e312-openshift-default-65c69cb84c Created pod: e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx 43m Normal ScalingReplicaSet deployment/e2e-inf-a-a4e312-openshift-default Scaled up replica set e2e-inf-a-a4e312-openshift-default-65c69cb84c from 1 to 2 43m Normal Created pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks Container created 43m Normal AddedInterface pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks Add eth0 [10.133.0.179/23] from ovn-kubernetes 43m Normal Started pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks Container started 43m Normal Pulled pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-785d26-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-785d26-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning ProbeError pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-785d26-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-785d26-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-754e0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-754e0a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Killing pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Deleted pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-htmwt 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-fd1c91-openshift-default New size: 4; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-fd1c91-openshift-default Scaled down replica set e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff from 5 to 4 42m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default Scaled down replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 6 to 5 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 5; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Deleted pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Deleted pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default New size: 5; reason: All metrics below target 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 5; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default Scaled down replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 6 to 5 42m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default Scaled down replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 6 to 5 42m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled down replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 6 to 5 42m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Deleted pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-6thpk 42m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mfft9 Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Deleted pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-6528w 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Deleted pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-kztj4 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Deleted pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 5; reason: All metrics below target 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 5; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-86j6n Stopping container istio-proxy 42m Normal Pulled pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk Add eth0 [10.132.0.195/23] from ovn-kubernetes 42m Normal Created pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk Container created 42m Normal Started pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk Container started 42m Normal Pulled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Add eth0 [10.132.0.196/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r Add eth0 [10.134.0.192/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g Add eth0 [10.133.0.180/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Readiness probe error: Get "http://10.132.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Created pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g Container created 42m Normal Created pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 Container created 42m Normal AddedInterface pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Add eth0 [10.132.0.197/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Container created 42m Normal Started pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Container started 42m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-lljdl Readiness probe failed: Get "http://10.132.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal Started pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g Container started 42m Normal Pulled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 Add eth0 [10.133.0.181/23] from ovn-kubernetes 42m Normal Started pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 Container started 42m Normal AddedInterface pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb Add eth0 [10.134.0.193/23] from ovn-kubernetes 42m Normal Created pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r Container created 42m Normal Pulled pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb Container created 42m Normal Started pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r Container started 42m Normal Started pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb Container started 42m Warning ProbeError pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk Startup probe error: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk Startup probe failed: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-7c892a Scaled up replica set payload-processing-e2e-inf-b-7c892a-58845567f5 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-7c892a Scaled up replica set payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9 Error creating: pods "payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-7c892a: serviceaccount "payload-processing-e2e-inf-b-7c892a" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-7c892a-58845567f5 Created pod: payload-processing-e2e-inf-b-7c892a-58845567f5-kgksv 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-7c892a-58845567f5 Error creating: pods "payload-processing-e2e-inf-b-7c892a-58845567f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-7c892a: serviceaccount "payload-processing-e2e-inf-b-7c892a" not found 42m Normal AddedInterface pod/payload-processing-e2e-inf-b-7c892a-58845567f5-kgksv Add eth0 [10.132.0.198/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9 Created pod: payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx 42m Normal Started pod/payload-processing-e2e-inf-b-7c892a-58845567f5-kgksv Container started 42m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 42m Normal Pulled pod/payload-processing-e2e-inf-b-7c892a-58845567f5-kgksv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r Startup probe error: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused... 42m Normal Created pod/payload-processing-e2e-inf-b-7c892a-58845567f5-kgksv Container created 42m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g Startup probe failed: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r Startup probe failed: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g Startup probe error: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb Startup probe failed: Get "http://10.134.0.193:15021/healthz/ready": dial tcp 10.134.0.193:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb Startup probe error: Get "http://10.134.0.193:15021/healthz/ready": dial tcp 10.134.0.193:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Startup probe failed: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Startup probe error: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused... 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-l6wqz 42m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 42m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-l6wqz Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75d7c785f from 10 to 8 42m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-msrfk Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-msrfk 42m Normal Started pod/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx Add eth0 [10.133.0.182/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx Container created 41m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default New size: 4; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Deleted pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-pf6kf 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-fd1c91-openshift-default New size: 3; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-fd1c91-openshift-default Scaled down replica set e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff from 4 to 3 41m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Deleted pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 41m Normal Killing pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-mbvk2 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 5; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Deleted pod: e2e-worker-w0-23843f-openshift-default-54cf54885-jstv5 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled down replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 6 to 5 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Deleted pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-r6b75 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default Scaled down replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 5 to 4 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 4; reason: All metrics below target 41m Normal Pulled pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Container created 41m Normal Started pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Container started 41m Warning ProbeError pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Startup probe error: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Startup probe failed: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-7c892a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-7c892a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v Stopping container payload-processing 40m Normal SuccessfulCreate replicaset/e2e-inf-b-9efeae-openshift-default-6df74d98cf Created pod: e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt 40m Normal ScalingReplicaSet deployment/e2e-inf-b-9efeae-openshift-default Scaled up replica set e2e-inf-b-9efeae-openshift-default-6df74d98cf from 2 to 3 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-9efeae-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 40m Normal Started pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Container started 40m Normal AddedInterface pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Add eth0 [10.133.0.192/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Container created 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-7c892a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-7c892a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default New size: 4; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Deleted pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk 40m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default New size: 3; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Deleted pod: e2e-worker-w0-23843f-openshift-default-54cf54885-88shr 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Deleted pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default New size: 3; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Deleted pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default Scaled down replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 5 to 4 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 4; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Deleted pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj 40m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-88shr Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Startup probe failed: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused 40m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-s75bw Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-gj6s5 40m Warning ProbeError pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Startup probe error: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused... 40m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75d7c785f from 8 to 7 40m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Stopping container istio-proxy 40m Normal Killing pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Stopping container istio-proxy 40m Normal Killing pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-4fnkf Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Readiness probe failed: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-mbv4q Readiness probe error: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning ProbeError pod/e2e-inf-a-03dfee-openshift-default-5ffbd99665-6zd65 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Readiness probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-mq4hj Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-gj6s5 Readiness probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Readiness probe error: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-t95xk Readiness probe failed: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning Unhealthy pod/payload-processing-e2e-inf-a-03dfee-6fb7d4f559-mhj7v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-03dfee-fc9fd5b8c-p6lgk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal Killing pod/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p Stopping container payload-processing 40m Normal Killing pod/payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg Stopping container payload-pre-processing 40m Normal ScalingReplicaSet deployment/e2e-inf-b-7c892a-openshift-default Scaled up replica set e2e-inf-b-7c892a-openshift-default-c8946d869 from 1 to 2 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-7c892a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz Stopping container payload-pre-processing 40m Normal SuccessfulCreate replicaset/e2e-inf-b-7c892a-openshift-default-c8946d869 Created pod: e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 40m Normal Killing pod/payload-processing-e2e-inf-a-b1fae7-6c4988f4d5-x9g5w Stopping container payload-processing 40m Normal AddedInterface pod/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 Add eth0 [10.133.0.197/23] from ovn-kubernetes 40m Normal Started pod/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 Container started 40m Normal Created pod/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 Container created 40m Normal Pulled pod/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Warning Unhealthy pod/payload-processing-e2e-inf-b-88eaf9-56b45b6577-m674p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-88eaf9-697ffc695-b9ds8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-8dcfa8-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-inf-a-754e0a-openshift-default-7c7cdd7487 Created pod: e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-754e0a-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-inf-a-754e0a-openshift-default-7c7cdd7487 Created pod: e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx 40m Normal SuccessfulCreate replicaset/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf Created pod: e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np 40m Normal ScalingReplicaSet deployment/e2e-inf-b-8dcfa8-openshift-default Scaled up replica set e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf from 2 to 3 40m Normal ScalingReplicaSet deployment/e2e-inf-a-754e0a-openshift-default Scaled up replica set e2e-inf-a-754e0a-openshift-default-7c7cdd7487 from 4 to 6 40m Normal Pulled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf Container started 40m Normal AddedInterface pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf Add eth0 [10.132.0.202/23] from ovn-kubernetes 40m Normal Created pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx Container created 40m Normal AddedInterface pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx Add eth0 [10.132.0.203/23] from ovn-kubernetes 40m Normal Created pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf Container created 40m Normal Started pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx Container started 40m Normal AddedInterface pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np Add eth0 [10.133.0.198/23] from ovn-kubernetes 40m Normal Created pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np Container created 40m Normal Started pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np Container started 40m Normal Killing pod/payload-processing-e2e-inf-a-a4e312-7b78fb8665-b9pn6 Stopping container payload-processing 40m Warning Unhealthy pod/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 Startup probe failed: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j Stopping container payload-pre-processing 40m Warning ProbeError pod/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 Startup probe error: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused... 40m Normal Killing pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-6sg8v Stopping container istio-proxy 40m Normal Killing pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-j699n Stopping container istio-proxy 40m Warning ProbeError pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np Startup probe error: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused... 40m Normal Killing pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np Startup probe failed: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused 40m Normal Killing pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-hqf9v Stopping container istio-proxy 40m Normal Killing pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-q68jj Stopping container istio-proxy 40m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx Startup probe error: Get "http://10.132.0.203:15021/healthz/ready": dial tcp 10.132.0.203:15021: connect: connection refused... 40m Normal Killing pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-55lch Stopping container istio-proxy 40m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf Startup probe error: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf Startup probe failed: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx Startup probe failed: Get "http://10.132.0.203:15021/healthz/ready": dial tcp 10.132.0.203:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-inf-a-278c04-openshift-default-9cb8c9ff5-rkpxs Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal Killing pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Stopping container istio-proxy 40m Normal Killing pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Stopping container istio-proxy 40m Normal AddedLabel gateway/e2e-shared-a-w3-c55cf1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-c55cf1 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Deleted pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default New size: 3; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-c55cf1-openshift-default Scaled up replica set e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87 from 0 to 1 40m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87 Created pod: e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 40m Warning Unhealthy pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Deleted pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh 40m Normal AddedInterface pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Add eth0 [10.133.0.199/23] from ovn-kubernetes 40m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-sp7bd Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-s8rhh Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal Killing pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-wt2tb Stopping container istio-proxy 40m Warning ProbeError pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal Killing pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-jxhzq Stopping container istio-proxy 40m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-inf-b-88eaf9-openshift-default-6c9fd5847f-jz2sb Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-g7x6w Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal Killing pod/e2e-inf-a-b1fae7-openshift-default-697b4b59f-9vvhg Stopping container istio-proxy 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-278c04-86645f6fbc-p5qmg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning Unhealthy pod/payload-processing-e2e-inf-a-278c04-6969d7fcb9-252tn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-c55cf1 Scaled up replica set payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c Created pod: payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-b1fae7-fbbcfb98c-bvdgz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f Created pod: payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f Error creating: pods "payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-c55cf1: serviceaccount "payload-processing-e2e-shared-a-w3-c55cf1" not found 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc Stopping container payload-processing 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c Error creating: pods "payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-c55cf1: serviceaccount "payload-processing-e2e-shared-a-w3-c55cf1" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-c55cf1 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c from 0 to 1 39m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg Add eth0 [10.134.0.195/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 Container started 39m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 Container created 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 Add eth0 [10.134.0.194/23] from ovn-kubernetes 39m Normal Started pod/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg Container started 39m Normal Created pod/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg Container created 39m Normal Pulled pod/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Killing pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Stopping container istio-proxy 39m Normal Killing pod/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm Stopping container payload-processing 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-a4e312-74585c8467-r5q8j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z Stopping container payload-pre-processing 39m Warning ProbeError pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-w9f5h Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning Unhealthy pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-inf-a-a4e312-openshift-default-65c69cb84c-t2vcx Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default New size: 3; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Deleted pod: e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs 39m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Stopping container istio-proxy 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-9efeae-699f495dc5-bh64l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/payload-processing-e2e-inf-b-d45f6c-59d8bb7487-tcsb7 Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg Stopping container payload-pre-processing 39m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Readiness probe failed: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-vh6zs Readiness probe error: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/payload-processing-e2e-inf-b-9efeae-6d89d89df7-7trdc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 Stopping container payload-processing 39m Normal Killing pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4g772 Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Stopping container istio-proxy 39m Warning ProbeError pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-4xf5z Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning Unhealthy pod/payload-processing-e2e-inf-b-f8c3f7-bdc4d8f7-75mhm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning ProbeError pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-inf-b-9efeae-openshift-default-6df74d98cf-mhkkt Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-f8c3f7-d97564b6c-z2z2z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/e2e-inf-a-1e8cf0-openshift-default-68f469b489-w6rnb Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-46xks Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Stopping container istio-proxy 39m Warning Unhealthy pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-inf-b-f8c3f7-openshift-default-78b8d845bc-8z2f2 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-d45f6c-55b6bc85f9-776mg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-1e8cf0-69f79d459f-4lmcb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning Unhealthy pod/payload-processing-e2e-inf-a-1e8cf0-56bb54564c-t5hz7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Stopping container istio-proxy 39m Warning ProbeError pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-6vqjp Stopping container istio-proxy 39m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 Created pod: e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 39m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e618e1-openshift-default Scaled up replica set e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 from 0 to 1 39m Normal AddedLabel gateway/e2e-shared-a-w4-e618e1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-e618e1 39m Normal AddedInterface pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Add eth0 [10.133.0.201/23] from ovn-kubernetes 39m Warning ProbeError pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-inf-b-d45f6c-openshift-default-7b5486ffcc-q2mtq Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/payload-processing-e2e-inf-b-8dcfa8-86d859f49f-j5f54 Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv Stopping container payload-pre-processing 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-e618e1 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759 Created pod: payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-e618e1 Scaled up replica set payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c Created pod: payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-e618e1: serviceaccount "payload-processing-e2e-shared-a-w4-e618e1" not found 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c Error creating: pods "payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-e618e1: serviceaccount "payload-processing-e2e-shared-a-w4-e618e1" not found 39m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 Add eth0 [10.134.0.197/23] from ovn-kubernetes 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd Add eth0 [10.134.0.198/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 Container created 39m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 Container started 39m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd Container created 39m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd Container started 39m Normal AddedLabel gateway/e2e-shared-b-w3-ae4ce2 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-ae4ce2 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f Created pod: e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 39m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-ae4ce2-openshift-default Scaled up replica set e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f from 0 to 1 39m Normal AddedInterface pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Add eth0 [10.133.0.203/23] from ovn-kubernetes 39m Normal Killing pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-z55xk Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-cz2np Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-ae4ce2 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc from 0 to 1 39m Normal Killing pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Stopping container istio-proxy 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc Error creating: pods "payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-ae4ce2: serviceaccount "payload-processing-e2e-shared-b-w3-ae4ce2" not found 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7 Error creating: pods "payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-ae4ce2: serviceaccount "payload-processing-e2e-shared-b-w3-ae4ce2" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7 Created pod: payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-h7nqs 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-ae4ce2 Scaled up replica set payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7 from 0 to 1 38m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Pulled pod/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-h7nqs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-h7nqs Add eth0 [10.134.0.199/23] from ovn-kubernetes 38m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd Container started 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc Created pod: payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd Add eth0 [10.134.0.200/23] from ovn-kubernetes 38m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd Container created 38m Warning Unhealthy pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal Started pod/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-h7nqs Container started 38m Normal Created pod/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-h7nqs Container created 38m Warning ProbeError pod/e2e-inf-b-8dcfa8-openshift-default-b5bfd9bf-9v4dz Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Started pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Container started 38m Normal Created pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Container created 38m Normal Pulled pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Add eth0 [10.133.0.205/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-inf-b-d8c398-openshift-default Scaled up replica set e2e-inf-b-d8c398-openshift-default-65cfb69fd7 from 1 to 2 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-d8c398-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-inf-b-d8c398-openshift-default-65cfb69fd7 Created pod: e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-8dcfa8-68fb9b44f5-6dmlv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning ProbeError pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Startup probe error: Get "http://10.133.0.205:15021/healthz/ready": dial tcp 10.133.0.205:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Startup probe failed: Get "http://10.133.0.205:15021/healthz/ready": dial tcp 10.133.0.205:15021: connect: connection refused 38m Normal Killing pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff Deleted pod: e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-lvxjm 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-fd1c91-openshift-default Scaled down replica set e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff from 3 to 2 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-fd1c91-openshift-default New size: 2; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Deleted pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t 38m Normal AddedLabel gateway/e2e-shared-b-w4-d9c048 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-d9c048 38m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 Created pod: e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 38m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-g4z7t Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-d9c048-openshift-default Scaled up replica set e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 from 0 to 1 38m Normal AddedInterface pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Add eth0 [10.132.0.214/23] from ovn-kubernetes 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5 Error creating: pods "payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-d9c048: serviceaccount "payload-processing-e2e-shared-b-w4-d9c048" not found 38m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp Add eth0 [10.133.0.208/23] from ovn-kubernetes 38m Normal Pulled pod/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-d9c048 Scaled up replica set payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-d9c048 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc from 0 to 1 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn Add eth0 [10.133.0.209/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc Created pod: payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5 Created pod: payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc Error creating: pods "payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-d9c048: serviceaccount "payload-processing-e2e-shared-b-w4-d9c048" not found 38m Normal Killing pod/payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 Stopping container payload-processing 38m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn Container created 38m Normal Killing pod/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc Stopping container payload-pre-processing 38m Normal Started pod/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp Container started 38m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn Container started 38m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp Container created 38m Normal Created pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Container created 38m Normal Pulled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e618e1-openshift-default Scaled up replica set e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 from 1 to 2 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-e618e1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 Created pod: e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t 38m Normal Started pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Container started 38m Normal Pulled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Container started 38m Normal Created pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Container created 38m Normal AddedInterface pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Add eth0 [10.132.0.215/23] from ovn-kubernetes 38m Warning ProbeError pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Readiness probe error: Get "http://10.133.0.199:15021/healthz/ready": read tcp 10.133.0.2:43698->10.133.0.199:15021: read: connection reset by peer... 38m Warning Unhealthy pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Readiness probe failed: Get "http://10.133.0.199:15021/healthz/ready": read tcp 10.133.0.2:43698->10.133.0.199:15021: read: connection reset by peer 38m Normal Started pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Container started 38m Normal Pulled pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Container created 38m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Startup probe failed: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused 38m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Startup probe failed: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Startup probe error: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Startup probe error: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Startup probe failed: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Startup probe error: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused... 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-c55cf1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-c55cf1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-e618e1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-f5c1ab-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f Created pod: e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-f5c1ab-openshift-default Scaled up replica set e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f from 1 to 2 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-e618e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Pulled pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/e2e-inf-a-754e0a-openshift-default-7c7cdd7487 Created pod: e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-754e0a-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 38m Normal AddedInterface pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 Add eth0 [10.132.0.217/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-inf-a-754e0a-openshift-default Scaled up replica set e2e-inf-a-754e0a-openshift-default-7c7cdd7487 from 6 to 8 38m Normal Created pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 Container created 38m Normal Started pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 Container started 38m Normal SuccessfulCreate replicaset/e2e-inf-a-754e0a-openshift-default-7c7cdd7487 Created pod: e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl 37m Normal Pulled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk Add eth0 [10.132.0.218/23] from ovn-kubernetes 37m Normal Created pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk Container created 37m Normal Started pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk Container started 37m Warning Unhealthy pod/payload-processing-e2e-inf-a-785d26-6467945f49-6xsr7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal AddedInterface pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl Add eth0 [10.134.0.203/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl Container created 37m Normal Started pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl Container started 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-785d26-589cf5bf5f-fmrnc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning ProbeError pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 Startup probe error: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl Startup probe error: Get "http://10.134.0.203:15021/healthz/ready": dial tcp 10.134.0.203:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl Startup probe failed: Get "http://10.134.0.203:15021/healthz/ready": dial tcp 10.134.0.203:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 Startup probe failed: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk Startup probe failed: Get "http://10.132.0.218:15021/healthz/ready": dial tcp 10.132.0.218:15021: connect: connection refused 37m Normal Killing pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Stopping container istio-proxy 37m Warning ProbeError pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk Startup probe error: Get "http://10.132.0.218:15021/healthz/ready": dial tcp 10.132.0.218:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-inf-a-785d26-openshift-default-74465b794c-qg2rb Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-c55cf1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-c55cf1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-e618e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-e618e1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Killing pod/payload-processing-e2e-inf-b-d8c398-546ff884f-pr754 Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc Stopping container payload-pre-processing 37m Normal Killing pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Stopping container istio-proxy 37m Warning Unhealthy pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-l26k2 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-d8c398-69fd677f-s95lc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-inf-b-d8c398-openshift-default-65cfb69fd7-jgxn5 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Readiness probe failed: Get "http://10.133.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Readiness probe error: Get "http://10.133.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Normal Pulled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd Container created 37m Normal Started pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd Container started 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 Created pod: e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 Created pod: e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 37m Normal Created pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Container created 37m Normal Started pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Container started 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-a891ce-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 37m Normal Pulled pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-a891ce-openshift-default Scaled up replica set e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 from 1 to 3 37m Normal Pulled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h Container started 37m Normal Created pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h Container created 37m Normal AddedInterface pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h Add eth0 [10.132.0.220/23] from ovn-kubernetes 37m Normal Created pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Container created 37m Normal AddedInterface pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Add eth0 [10.133.0.213/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Container started 37m Warning Unhealthy pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd Startup probe failed: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd Startup probe error: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Startup probe error: Get "http://10.133.0.203:15021/healthz/ready": dial tcp 10.133.0.203:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Startup probe failed: Get "http://10.133.0.203:15021/healthz/ready": dial tcp 10.133.0.203:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Startup probe error: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Startup probe failed: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h Startup probe error: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h Startup probe failed: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-435728-openshift-default Scaled up replica set e2e-shared-b-w1-435728-openshift-default-5f778d7cf7 from 1 to 2 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7 Created pod: e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-435728-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal Started pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 Container started 37m Normal Created pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 Container created 37m Normal Pulled pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 Add eth0 [10.132.0.221/23] from ovn-kubernetes 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 Created pod: e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs 37m Normal Pulled pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs Add eth0 [10.132.0.222/23] from ovn-kubernetes 37m Normal Started pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs Container started 37m Normal Created pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs Container created 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-c55cf1-openshift-default Scaled up replica set e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87 from 1 to 2 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-c55cf1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87 Created pod: e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 37m Normal Pulled pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 Container started 37m Normal AddedInterface pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 Add eth0 [10.132.0.223/23] from ovn-kubernetes 37m Normal Created pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 Container created 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-a891ce-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning ProbeError pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 Startup probe error: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 Startup probe failed: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-a891ce-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning Unhealthy pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs Startup probe failed: Get "http://10.132.0.222:15021/healthz/ready": dial tcp 10.132.0.222:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs Startup probe error: Get "http://10.132.0.222:15021/healthz/ready": dial tcp 10.132.0.222:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 Startup probe error: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 Startup probe failed: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-ae4ce2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-ae4ce2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-a891ce-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-a891ce-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Readiness probe failed: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Readiness probe error: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal Created pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Container created 36m Normal Pulled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Container started 36m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-d9c048-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-d9c048-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647 Created pod: e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-4bac5a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-4bac5a-openshift-default Scaled up replica set e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647 from 1 to 2 35m Normal Created pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 Container created 35m Normal AddedInterface pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 Add eth0 [10.133.0.217/23] from ovn-kubernetes 35m Normal Started pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 Container started 35m Normal Pulled pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning Unhealthy pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 Startup probe failed: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 Startup probe error: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused... 35m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 Container started 35m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 Container created 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-d9c048-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-d9c048-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused... 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-ae2e60-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-ae2e60-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 35m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-l4p8z Stopping container discovery 35m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-l4p8z 35m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 35m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 Container created 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae2e60-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk 35m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 Add eth0 [10.132.0.225/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Container started 35m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Container started 35m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Container created 35m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae2e60-openshift-default Scaled up replica set e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 from 1 to 4 35m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Container created 35m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 Container started 35m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Add eth0 [10.133.0.220/23] from ovn-kubernetes 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 35m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Add eth0 [10.134.0.205/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Startup probe error: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 Startup probe error: Get "http://10.132.0.225:15021/healthz/ready": dial tcp 10.132.0.225:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Startup probe failed: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Startup probe failed: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 Startup probe failed: Get "http://10.132.0.225:15021/healthz/ready": dial tcp 10.132.0.225:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Startup probe error: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused... 35m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Container created 35m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Container created 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l 35m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Add eth0 [10.133.0.222/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Add eth0 [10.134.0.206/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Add eth0 [10.133.0.221/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Container created 35m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Container started 35m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Add eth0 [10.132.0.228/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae2e60-openshift-default Scaled up replica set e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 from 4 to 8 35m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Container started 35m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Container started 35m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae2e60-openshift-default New size: 8; reason: 35m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Container started 35m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Container created 34m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Startup probe failed: Get "http://10.134.0.206:15021/healthz/ready": dial tcp 10.134.0.206:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Startup probe error: Get "http://10.134.0.206:15021/healthz/ready": dial tcp 10.134.0.206:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Startup probe failed: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Startup probe error: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Startup probe error: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Startup probe failed: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Startup probe failed: Get "http://10.133.0.222:15021/healthz/ready": dial tcp 10.133.0.222:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Startup probe error: Get "http://10.133.0.222:15021/healthz/ready": dial tcp 10.133.0.222:15021: connect: connection refused... 34m Normal Killing pod/payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb Stopping container payload-pre-processing 34m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Add eth0 [10.134.0.207/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae2e60-openshift-default Scaled up replica set e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 from 8 to 10 34m Normal AddedInterface pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Add eth0 [10.132.0.229/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Container started 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Created pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae2e60-openshift-default New size: 10; reason: 34m Normal Pulled pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Container created 34m Normal Started pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Container started 34m Normal Created pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Container created 34m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Startup probe error: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Startup probe failed: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Startup probe error: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Startup probe failed: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e618e1-openshift-default Scaled up replica set e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 from 2 to 3 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 Created pod: e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-e618e1-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 34m Normal AddedInterface pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Add eth0 [10.134.0.208/23] from ovn-kubernetes 34m Normal Started pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Container started 34m Normal Created pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Container created 34m Normal Pulled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 Created pod: e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs 34m Normal Killing pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-lv679 Stopping container istio-proxy 34m Normal Pulled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-754e0a-85f75ff745-b8jz8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-754e0a-7858857447-zl6nb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-b594r Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-p5bxx Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dxl4g Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-dl2c9 Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-sljnl Stopping container istio-proxy 34m Normal Created pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Container created 34m Normal Killing pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-q4mvk Stopping container istio-proxy 34m Normal AddedInterface pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Add eth0 [10.132.0.230/23] from ovn-kubernetes 34m Normal Killing pod/e2e-inf-a-754e0a-openshift-default-7c7cdd7487-ktzsf Stopping container istio-proxy 34m Normal Started pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Container started 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-d9c048-openshift-default Scaled up replica set e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 from 1 to 2 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 Created pod: e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-d9c048-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal AddedInterface pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk Add eth0 [10.133.0.223/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk Container started 34m Normal Created pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk Container created 34m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Startup probe failed: Get "http://10.134.0.208:15021/healthz/ready": dial tcp 10.134.0.208:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Startup probe error: Get "http://10.134.0.208:15021/healthz/ready": dial tcp 10.134.0.208:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Startup probe error: Get "http://10.132.0.230:15021/healthz/ready": dial tcp 10.132.0.230:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Startup probe failed: Get "http://10.132.0.230:15021/healthz/ready": dial tcp 10.132.0.230:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk Startup probe failed: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk Startup probe error: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused... 34m Normal Killing pod/payload-processing-e2e-inf-b-7c892a-58845567f5-kgksv Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx Stopping container payload-pre-processing 34m Warning Unhealthy pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal Killing pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Stopping container istio-proxy 34m Warning ProbeError pod/e2e-inf-b-7c892a-openshift-default-c8946d869-pr4j9 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/e2e-inf-b-7c892a-openshift-default-c8946d869-qv6n9 Stopping container istio-proxy 34m Normal SuccessfulCreate replicaset/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7 Created pod: e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs 34m Normal ScalingReplicaSet deployment/e2e-ipp-a-1f2781-openshift-default Scaled up replica set e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7 from 0 to 1 34m Normal AddedLabel gateway/e2e-ipp-a-1f2781 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-1f2781 34m Normal AddedInterface pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Add eth0 [10.133.0.224/23] from ovn-kubernetes 34m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-23843f-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-8b083a-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w1-8b083a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-8b083a-openshift-default-f965c5bc5 from 3 to 2 34m Normal ScalingReplicaSet deployment/e2e-worker-w2-aeb292-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-aeb292-openshift-default-58cb568b9f from 3 to 2 34m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-aeb292-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-996c32-openshift-default-58dc84fb9 Deleted pod: e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-8b083a-openshift-default-f965c5bc5 Deleted pod: e2e-worker-w1-8b083a-openshift-default-f965c5bc5-xz8h6 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-996c32-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/e2e-worker-w4-996c32-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-996c32-openshift-default-58dc84fb9 from 3 to 2 34m Normal SuccessfulDelete replicaset/e2e-worker-w0-23843f-openshift-default-54cf54885 Deleted pod: e2e-worker-w0-23843f-openshift-default-54cf54885-qvsmr 34m Normal ScalingReplicaSet deployment/e2e-worker-w0-23843f-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-23843f-openshift-default-54cf54885 from 3 to 2 34m Normal SuccessfulDelete replicaset/e2e-worker-w2-aeb292-openshift-default-58cb568b9f Deleted pod: e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 34m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Deleted pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc 34m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75d7c785f from 7 to 6 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-tbx78 34m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-tbx78 Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479 Error creating: pods "payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-1f2781: serviceaccount "payload-processing-e2e-ipp-a-1f2781" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479 Created pod: payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-894jp 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-1f2781 Scaled up replica set payload-pre-processing-e2e-ipp-a-1f2781-57c8788479 from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-1f2781 Scaled up replica set payload-processing-e2e-ipp-a-1f2781-76cd6d497f from 0 to 1 33m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-1f2781-76cd6d497f Error creating: pods "payload-processing-e2e-ipp-a-1f2781-76cd6d497f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-1f2781: serviceaccount "payload-processing-e2e-ipp-a-1f2781" not found 33m Normal Created pod/payload-processing-e2e-ipp-a-1f2781-76cd6d497f-sbqjg Container created 33m Normal Pulled pod/payload-processing-e2e-ipp-a-1f2781-76cd6d497f-sbqjg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-processing-e2e-ipp-a-1f2781-76cd6d497f-sbqjg Add eth0 [10.132.0.233/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-894jp Container started 33m Normal Created pod/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-894jp Container created 33m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-894jp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-894jp Add eth0 [10.132.0.232/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-1f2781-76cd6d497f Created pod: payload-processing-e2e-ipp-a-1f2781-76cd6d497f-sbqjg 33m Normal Started pod/payload-processing-e2e-ipp-a-1f2781-76cd6d497f-sbqjg Container started 33m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-n7xkc Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-ghgbn Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-k8p48 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-7c892a-98d9d78c9-qrnvx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/e2e-worker-w6-8730d5-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-8730d5-openshift-default-59d8dd4587 from 3 to 2 33m Normal SuccessfulDelete replicaset/e2e-worker-w6-8730d5-openshift-default-59d8dd4587 Deleted pod: e2e-worker-w6-8730d5-openshift-default-59d8dd4587-txzvm 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-8730d5-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 33m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/e2e-worker-w3-23814d-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-23814d-openshift-default-775b6897bc from 4 to 3 33m Normal SuccessfulDelete replicaset/e2e-worker-w3-23814d-openshift-default-775b6897bc Deleted pod: e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23814d-openshift-default (combined from similar events): New size: 3; reason: All metrics below target 33m Warning ProbeError pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Readiness probe error: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bjc82 Readiness probe failed: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-1f2781-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-1f2781-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Killing pod/payload-processing-e2e-ipp-a-1f2781-76cd6d497f-sbqjg Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-ipp-a-1f2781-57c8788479-894jp Stopping container payload-pre-processing 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-1f2781-857bf89d44 Created pod: payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf Error creating: pods "payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-1f2781: serviceaccount "payload-processing-e2e-ipp-a-1f2781" not found 33m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-1f2781-857bf89d44 Error creating: pods "payload-processing-e2e-ipp-a-1f2781-857bf89d44-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-1f2781: serviceaccount "payload-processing-e2e-ipp-a-1f2781" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf Created pod: payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-1f2781 Scaled up replica set payload-processing-e2e-ipp-a-1f2781-857bf89d44 from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-1f2781 Scaled up replica set payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf from 0 to 1 33m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq Add eth0 [10.134.0.209/23] from ovn-kubernetes 33m Normal Pulling pod/payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 33m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 33m Normal AddedInterface pod/payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj Add eth0 [10.134.0.210/23] from ovn-kubernetes 33m Normal Created pod/payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj Container created 33m Normal Started pod/payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj Container started 33m Normal Pulled pod/payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.286s (2.286s including waiting). Image size: 166620835 bytes. 33m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.271s (2.271s including waiting). Image size: 166620835 bytes. 33m Normal Created pod/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq Container created 33m Normal Started pod/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq Container started 33m Normal ScalingReplicaSet deployment/e2e-worker-w5-5bbf75-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 from 3 to 2 33m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65 Deleted pod: e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-p2x8s 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-5bbf75-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 33m Warning ProbeError pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Readiness probe error: Get "http://10.133.0.224:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75d7c785f from 6 to 5 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-9th7f 33m Warning Unhealthy pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Readiness probe failed: Get "http://10.133.0.224:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-9th7f Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 32m Normal Started pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Container started 32m Normal Created pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Container created 32m Normal Pulled pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning Unhealthy pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Startup probe failed: Get "http://10.133.0.224:15021/healthz/ready": dial tcp 10.133.0.224:15021: connect: connection refused 32m Warning ProbeError pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Startup probe error: Get "http://10.133.0.224:15021/healthz/ready": dial tcp 10.133.0.224:15021: connect: connection refused... 32m Normal AddedLabel gateway/e2e-ipp-b-01afb5 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-01afb5 32m Normal ScalingReplicaSet deployment/e2e-ipp-b-01afb5-openshift-default Scaled up replica set e2e-ipp-b-01afb5-openshift-default-5464b8fc9 from 0 to 1 32m Normal SuccessfulCreate replicaset/e2e-ipp-b-01afb5-openshift-default-5464b8fc9 Created pod: e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n 32m Normal AddedInterface pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Add eth0 [10.133.0.225/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-f5c1ab-openshift-default Scaled up replica set e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f from 2 to 3 32m Normal Pulled pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr Container created 32m Normal Started pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr Container started 32m Normal AddedInterface pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr Add eth0 [10.134.0.211/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f Created pod: e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-f5c1ab-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955 Created pod: payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-85np2 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955 Error creating: pods "payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-01afb5: serviceaccount "payload-processing-e2e-ipp-b-01afb5" not found 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-01afb5-7956f95fdf Error creating: pods "payload-processing-e2e-ipp-b-01afb5-7956f95fdf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-01afb5: serviceaccount "payload-processing-e2e-ipp-b-01afb5" not found 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-01afb5 Scaled up replica set payload-processing-e2e-ipp-b-01afb5-7956f95fdf from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-01afb5-7956f95fdf Created pod: payload-processing-e2e-ipp-b-01afb5-7956f95fdf-4sgpr 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-01afb5 Scaled up replica set payload-pre-processing-e2e-ipp-b-01afb5-b944c7955 from 0 to 1 32m Normal AddedInterface pod/payload-processing-e2e-ipp-b-01afb5-7956f95fdf-4sgpr Add eth0 [10.133.0.226/23] from ovn-kubernetes 32m Normal Started pod/payload-processing-e2e-ipp-b-01afb5-7956f95fdf-4sgpr Container started 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-85np2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Pulled pod/payload-processing-e2e-ipp-b-01afb5-7956f95fdf-4sgpr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-processing-e2e-ipp-b-01afb5-7956f95fdf-4sgpr Container created 32m Normal Started pod/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-85np2 Container started 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-85np2 Add eth0 [10.132.0.238/23] from ovn-kubernetes 32m Normal Created pod/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-85np2 Container created 32m Warning Unhealthy pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr Startup probe failed: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr Startup probe error: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused... 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-1f2781-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-1f2781-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal AddedLabel gateway/e2e-ipp-a-26cd59 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-26cd59 32m Normal SuccessfulCreate replicaset/e2e-ipp-a-26cd59-openshift-default-74bbc85d85 Created pod: e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq 32m Normal ScalingReplicaSet deployment/e2e-ipp-a-26cd59-openshift-default Scaled up replica set e2e-ipp-a-26cd59-openshift-default-74bbc85d85 from 0 to 1 32m Normal AddedInterface pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Add eth0 [10.132.0.240/23] from ovn-kubernetes 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9 Error creating: pods "payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-26cd59: serviceaccount "payload-processing-e2e-ipp-a-26cd59" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9 Created pod: payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-j7z8j 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f Created pod: payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-hfrgd 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-26cd59 Scaled up replica set payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f Error creating: pods "payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-26cd59: serviceaccount "payload-processing-e2e-ipp-a-26cd59" not found 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-26cd59 Scaled up replica set payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9 from 0 to 1 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-hfrgd Add eth0 [10.133.0.227/23] from ovn-kubernetes 32m Normal Pulled pod/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-j7z8j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-j7z8j Add eth0 [10.133.0.228/23] from ovn-kubernetes 32m Normal Created pod/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-j7z8j Container created 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-hfrgd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-hfrgd Container created 32m Normal Started pod/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-j7z8j Container started 32m Normal Started pod/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-hfrgd Container started 32m Normal Killing pod/payload-processing-e2e-ipp-b-01afb5-7956f95fdf-4sgpr Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-ipp-b-01afb5-b944c7955-85np2 Stopping container payload-pre-processing 31m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75d7c785f from 5 to 4 31m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-6qm4f 31m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-6qm4f Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-e1d060-openshift-default New size: 1; reason: All metrics below target 31m Normal Killing pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 Deleted pod: e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-mrdjs 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e1d060-openshift-default Scaled down replica set e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 from 2 to 1 31m Normal Killing pod/payload-processing-e2e-ipp-a-26cd59-79c7d4d9c9-j7z8j Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-ipp-a-26cd59-554cf8bc5f-hfrgd Stopping container payload-pre-processing 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-01afb5-db5b465fd Error creating: pods "payload-processing-e2e-ipp-b-01afb5-db5b465fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-01afb5: serviceaccount "payload-processing-e2e-ipp-b-01afb5" not found 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-01afb5 Scaled up replica set payload-processing-e2e-ipp-b-01afb5-db5b465fd from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7 Created pod: payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7 Error creating: pods "payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-01afb5: serviceaccount "payload-processing-e2e-ipp-b-01afb5" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-01afb5-db5b465fd Created pod: payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-01afb5 Scaled up replica set payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7 from 0 to 1 31m Normal Pulling pod/payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 31m Normal AddedInterface pod/payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg Add eth0 [10.133.0.230/23] from ovn-kubernetes 31m Normal Pulling pod/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv Add eth0 [10.133.0.229/23] from ovn-kubernetes 31m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Readiness probe error: Get "http://10.132.0.229:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 31m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Readiness probe failed: Get "http://10.132.0.229:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 8.283s (8.283s including waiting). Image size: 166620835 bytes. 31m Normal Pulled pod/payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 8.032s (8.032s including waiting). Image size: 166620835 bytes. 31m Normal Created pod/payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg Container created 31m Normal Started pod/payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg Container started 31m Normal Created pod/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv Container created 31m Normal Started pod/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv Container started 31m Normal Created pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Container created 31m Normal Pulled pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Container started 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-e1d060-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e1d060-openshift-default Scaled up replica set e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 from 1 to 2 31m Normal AddedInterface pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Add eth0 [10.132.0.246/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 Created pod: e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd 31m Normal ScalingReplicaSet deployment/e2e-ipp-b-01afb5-openshift-default Scaled up replica set e2e-ipp-b-01afb5-openshift-default-5464b8fc9 from 1 to 3 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-01afb5-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 31m Normal AddedInterface pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l Add eth0 [10.132.0.247/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal SuccessfulCreate replicaset/e2e-ipp-b-01afb5-openshift-default-5464b8fc9 Created pod: e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l 31m Normal SuccessfulCreate replicaset/e2e-ipp-b-01afb5-openshift-default-5464b8fc9 Created pod: e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv 31m Normal AddedInterface pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv Add eth0 [10.134.0.213/23] from ovn-kubernetes 31m Normal Created pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv Container created 31m Normal Pulled pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv Container started 31m Normal Started pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l Container started 31m Normal Created pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l Container created 31m Warning Unhealthy pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Readiness probe failed: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused 31m Warning ProbeError pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Readiness probe error: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused... 31m Normal Started pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Container started 31m Normal Created pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Container created 31m Normal Pulled pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5 Error creating: pods "payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-26cd59: serviceaccount "payload-processing-e2e-ipp-a-26cd59" not found 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-26cd59 Scaled up replica set payload-processing-e2e-ipp-a-26cd59-84899b8c48 from 0 to 1 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-26cd59-84899b8c48 Error creating: pods "payload-processing-e2e-ipp-a-26cd59-84899b8c48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-26cd59: serviceaccount "payload-processing-e2e-ipp-a-26cd59" not found 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5 Created pod: payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-z4pwr 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-26cd59-84899b8c48 Created pod: payload-processing-e2e-ipp-a-26cd59-84899b8c48-bgz8c 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-26cd59 Scaled up replica set payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5 from 0 to 1 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-z4pwr Add eth0 [10.133.0.236/23] from ovn-kubernetes 31m Normal Started pod/payload-processing-e2e-ipp-a-26cd59-84899b8c48-bgz8c Container started 31m Normal AddedInterface pod/payload-processing-e2e-ipp-a-26cd59-84899b8c48-bgz8c Add eth0 [10.133.0.235/23] from ovn-kubernetes 31m Normal Pulled pod/payload-processing-e2e-ipp-a-26cd59-84899b8c48-bgz8c Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-z4pwr Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-processing-e2e-ipp-a-26cd59-84899b8c48-bgz8c Container created 31m Normal Started pod/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-z4pwr Container started 31m Normal Created pod/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-z4pwr Container created 31m Warning Unhealthy pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv Startup probe failed: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Startup probe error: Get "http://10.132.0.246:15021/healthz/ready": dial tcp 10.132.0.246:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv Startup probe error: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Startup probe failed: Get "http://10.132.0.246:15021/healthz/ready": dial tcp 10.132.0.246:15021: connect: connection refused 31m Warning ProbeError pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l Startup probe error: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l Startup probe failed: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused 31m Warning Unhealthy pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Startup probe failed: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused 31m Warning ProbeError pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Startup probe error: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused... 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-ae4ce2-openshift-default Scaled up replica set e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f from 1 to 2 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-ae4ce2-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f Created pod: e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 31m Normal AddedInterface pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 Add eth0 [10.132.0.249/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 Container created 31m Normal Started pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 Container started 31m Normal Pulled pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-01afb5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-01afb5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-e618e1-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 31m Normal AddedInterface pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Add eth0 [10.133.0.239/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 Created pod: e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 Created pod: e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 Created pod: e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh 31m Normal AddedInterface pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Add eth0 [10.134.0.214/23] from ovn-kubernetes 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e618e1-openshift-default Scaled up replica set e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 from 3 to 6 31m Normal Pulled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Container created 31m Normal Started pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Container started 31m Normal Started pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Container started 31m Normal AddedInterface pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Add eth0 [10.132.0.251/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Container created 31m Normal Pulled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Pulled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal ScalingReplicaSet deployment/e2e-ipp-b-5f3eff-openshift-default Scaled up replica set e2e-ipp-b-5f3eff-openshift-default-78b5fb4875 from 0 to 1 31m Normal Created pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Container created 31m Normal AddedLabel gateway/e2e-ipp-b-5f3eff Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-5f3eff 31m Normal SuccessfulCreate replicaset/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875 Created pod: e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v 31m Normal Started pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Container started 31m Warning ProbeError pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 Startup probe error: Get "http://10.132.0.249:15021/healthz/ready": dial tcp 10.132.0.249:15021: connect: connection refused... 31m Normal AddedInterface pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Add eth0 [10.133.0.240/23] from ovn-kubernetes 31m Warning Unhealthy pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 Startup probe failed: Get "http://10.132.0.249:15021/healthz/ready": dial tcp 10.132.0.249:15021: connect: connection refused 31m Warning ProbeError pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Readiness probe error: Get "http://10.132.0.240:15021/healthz/ready": read tcp 10.132.0.2:58560->10.132.0.240:15021: read: connection reset by peer... 31m Warning Unhealthy pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Readiness probe failed: Get "http://10.132.0.240:15021/healthz/ready": read tcp 10.132.0.2:58560->10.132.0.240:15021: read: connection reset by peer 31m Normal Started pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Container started 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-d9c048-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 31m Normal Pulled pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Container created 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 Created pod: e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 Created pod: e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-d9c048-openshift-default Scaled up replica set e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 from 2 to 4 31m Normal Started pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Container started 31m Normal AddedInterface pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Add eth0 [10.132.0.252/23] from ovn-kubernetes 31m Normal AddedInterface pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Add eth0 [10.134.0.215/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Container created 31m Normal Pulled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Pulled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-5f3eff-7655b4885c Error creating: pods "payload-processing-e2e-ipp-b-5f3eff-7655b4885c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5f3eff: serviceaccount "payload-processing-e2e-ipp-b-5f3eff" not found 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9 Error creating: pods "payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5f3eff: serviceaccount "payload-processing-e2e-ipp-b-5f3eff" not found 31m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Startup probe error: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused... 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-5f3eff Scaled up replica set payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9 from 0 to 1 31m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Startup probe failed: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused 31m Normal Created pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Container created 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-5f3eff Scaled up replica set payload-processing-e2e-ipp-b-5f3eff-7655b4885c from 0 to 1 31m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Startup probe failed: Get "http://10.133.0.239:15021/healthz/ready": dial tcp 10.133.0.239:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Startup probe error: Get "http://10.133.0.239:15021/healthz/ready": dial tcp 10.133.0.239:15021: connect: connection refused... 31m Normal Started pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Container started 31m Normal Pulled pod/payload-processing-e2e-ipp-b-5f3eff-7655b4885c-k6dn9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-processing-e2e-ipp-b-5f3eff-7655b4885c-k6dn9 Container created 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9 Created pod: payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-2c4l7 31m Normal Started pod/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-2c4l7 Container started 31m Normal Started pod/payload-processing-e2e-ipp-b-5f3eff-7655b4885c-k6dn9 Container started 31m Normal Created pod/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-2c4l7 Container created 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-2c4l7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-5f3eff-7655b4885c Created pod: payload-processing-e2e-ipp-b-5f3eff-7655b4885c-k6dn9 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-2c4l7 Add eth0 [10.133.0.242/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-processing-e2e-ipp-b-5f3eff-7655b4885c-k6dn9 Add eth0 [10.133.0.243/23] from ovn-kubernetes 31m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Startup probe error: Get "http://10.132.0.251:15021/healthz/ready": dial tcp 10.132.0.251:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Startup probe failed: Get "http://10.132.0.251:15021/healthz/ready": dial tcp 10.132.0.251:15021: connect: connection refused 31m Warning ProbeError pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Startup probe error: Get "http://10.132.0.240:15021/healthz/ready": dial tcp 10.132.0.240:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Startup probe failed: Get "http://10.132.0.240:15021/healthz/ready": dial tcp 10.132.0.240:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Startup probe error: Get "http://10.134.0.215:15021/healthz/ready": dial tcp 10.134.0.215:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Startup probe failed: Get "http://10.134.0.215:15021/healthz/ready": dial tcp 10.134.0.215:15021: connect: connection refused 31m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Startup probe failed: Get "http://10.132.0.252:15021/healthz/ready": dial tcp 10.132.0.252:15021: connect: connection refused 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-01afb5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Startup probe error: Get "http://10.132.0.252:15021/healthz/ready": dial tcp 10.132.0.252:15021: connect: connection refused... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-01afb5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e618e1-openshift-default Scaled up replica set e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 from 6 to 7 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987 Created pod: e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-e618e1-openshift-default New size: 7; reason: All metrics below target 30m Normal Pulled pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Container created 30m Normal Started pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Container started 30m Normal AddedInterface pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Add eth0 [10.133.0.244/23] from ovn-kubernetes 30m Normal SuccessfulCreate replicaset/e2e-ipp-a-c173d8-openshift-default-6945698f95 Created pod: e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b 30m Normal AddedLabel gateway/e2e-ipp-a-c173d8 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-c173d8 30m Normal ScalingReplicaSet deployment/e2e-ipp-a-c173d8-openshift-default Scaled up replica set e2e-ipp-a-c173d8-openshift-default-6945698f95 from 0 to 1 30m Normal AddedInterface pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Add eth0 [10.133.0.246/23] from ovn-kubernetes 30m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Startup probe error: Get "http://10.133.0.244:15021/healthz/ready": dial tcp 10.133.0.244:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Startup probe failed: Get "http://10.133.0.244:15021/healthz/ready": dial tcp 10.133.0.244:15021: connect: connection refused 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c173d8 Scaled up replica set payload-processing-e2e-ipp-a-c173d8-8c8c9bf69 from 0 to 1 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c173d8 Scaled up replica set payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69 Created pod: payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f Created pod: payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-pgf8r 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f Error creating: pods "payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c173d8: serviceaccount "payload-processing-e2e-ipp-a-c173d8" not found 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69 Error creating: pods "payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c173d8: serviceaccount "payload-processing-e2e-ipp-a-c173d8" not found 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-pgf8r Container started 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-pgf8r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-pgf8r Container created 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-pgf8r Add eth0 [10.133.0.247/23] from ovn-kubernetes 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz Add eth0 [10.133.0.248/23] from ovn-kubernetes 30m Normal Pulled pod/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Created pod/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz Container created 30m Normal Started pod/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz Container started 30m Normal Killing pod/payload-processing-e2e-ipp-a-1f2781-857bf89d44-gqflj Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-ipp-a-1f2781-5d9d88cccf-2pbxq Stopping container payload-pre-processing 30m Normal SuccessfulCreate replicaset/e2e-ipp-a-c734f5-openshift-default-7f99c45bc Created pod: e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw 30m Normal AddedLabel gateway/e2e-ipp-a-c734f5 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-c734f5 30m Normal ScalingReplicaSet deployment/e2e-ipp-a-c734f5-openshift-default Scaled up replica set e2e-ipp-a-c734f5-openshift-default-7f99c45bc from 0 to 1 30m Normal AddedInterface pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Add eth0 [10.132.1.8/23] from ovn-kubernetes 30m Normal Killing pod/e2e-ipp-a-1f2781-openshift-default-6c5fbfd5f7-w7jcs Stopping container istio-proxy 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c Error creating: pods "payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c734f5: serviceaccount "payload-processing-e2e-ipp-a-c734f5" not found 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c734f5 Scaled up replica set payload-processing-e2e-ipp-a-c734f5-699f7df769 from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c734f5-699f7df769 Created pod: payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c Created pod: payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-nxcxx 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c734f5-699f7df769 Error creating: pods "payload-processing-e2e-ipp-a-c734f5-699f7df769-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c734f5: serviceaccount "payload-processing-e2e-ipp-a-c734f5" not found 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c734f5 Scaled up replica set payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c from 0 to 1 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-nxcxx Container started 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-nxcxx Container created 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-nxcxx Add eth0 [10.134.0.220/23] from ovn-kubernetes 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-nxcxx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Pulled pod/payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh Add eth0 [10.133.1.5/23] from ovn-kubernetes 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-26cd59-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal Started pod/payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh Container started 30m Normal Created pod/payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh Container created 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-26cd59-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal Killing pod/payload-pre-processing-e2e-ipp-b-5f3eff-6f85d65cf9-2c4l7 Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-ipp-b-5f3eff-7655b4885c-k6dn9 Stopping container payload-processing 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-cd9729-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-cd9729-openshift-default Scaled up replica set e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869 from 1 to 2 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869 Created pod: e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 30m Normal Started pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 Container started 30m Normal Pulled pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 Add eth0 [10.132.1.10/23] from ovn-kubernetes 30m Normal Created pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 Container created 30m Normal ScalingReplicaSet deployment/e2e-ipp-a-bd1c9f-openshift-default Scaled up replica set e2e-ipp-a-bd1c9f-openshift-default-74d884b45c from 0 to 1 30m Normal AddedLabel gateway/e2e-ipp-a-bd1c9f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-bd1c9f 30m Normal SuccessfulCreate replicaset/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c Created pod: e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 30m Normal AddedInterface pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Add eth0 [10.133.1.7/23] from ovn-kubernetes 30m Normal Killing pod/payload-processing-e2e-ipp-b-01afb5-db5b465fd-jkvfg Stopping container payload-processing 30m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-b-01afb5 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-01afb5: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-01afb5": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-01afb5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4c1d459b-df02-4cc1-b6f4-0d099517f0c8, UID in object meta: 30m Normal Killing pod/payload-pre-processing-e2e-ipp-b-01afb5-6fdd5b76b7-qggmv Stopping container payload-pre-processing 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-d9c048-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-d9c048-openshift-default Scaled up replica set e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 from 4 to 5 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57 Created pod: e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-bd1c9f Scaled up replica set payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79 from 0 to 1 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf Error creating: pods "payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-bd1c9f: serviceaccount "payload-processing-e2e-ipp-a-bd1c9f" not found 30m Normal Started pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Container started 30m Normal Pulled pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Add eth0 [10.133.1.8/23] from ovn-kubernetes 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-bd1c9f Scaled up replica set payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf Created pod: payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-qnnw8 30m Normal Created pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Container created 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79 Error creating: pods "payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-bd1c9f: serviceaccount "payload-processing-e2e-ipp-a-bd1c9f" not found 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79 Created pod: payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-v5wmk 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-v5wmk Add eth0 [10.133.1.9/23] from ovn-kubernetes 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-qnnw8 Add eth0 [10.133.1.10/23] from ovn-kubernetes 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-v5wmk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Warning ProbeError pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 Startup probe error: Get "http://10.132.1.10:15021/healthz/ready": dial tcp 10.132.1.10:15021: connect: connection refused... 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-v5wmk Container started 30m Normal Pulled pod/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-qnnw8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Warning Unhealthy pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 Startup probe failed: Get "http://10.132.1.10:15021/healthz/ready": dial tcp 10.132.1.10:15021: connect: connection refused 30m Normal Created pod/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-qnnw8 Container created 30m Normal Started pod/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-qnnw8 Container started 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-v5wmk Container created 30m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c173d8-56dc894d7f-pgf8r Stopping container payload-pre-processing 30m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Startup probe failed: Get "http://10.133.1.8:15021/healthz/ready": dial tcp 10.133.1.8:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Startup probe error: Get "http://10.133.1.8:15021/healthz/ready": dial tcp 10.133.1.8:15021: connect: connection refused... 30m Normal Killing pod/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz Stopping container payload-processing 30m Normal Killing pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-gtz5l Stopping container istio-proxy 30m Normal Killing pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-qbt9n Stopping container istio-proxy 30m Normal Killing pod/e2e-ipp-b-01afb5-openshift-default-5464b8fc9-pspcv Stopping container istio-proxy 29m Warning Unhealthy pod/payload-processing-e2e-ipp-a-c173d8-8c8c9bf69-d6xbz Readiness probe failed: timeout: failed to connect service "10.133.0.248:50052" within 1s: context deadline exceeded 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-5f3eff Scaled up replica set payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-5f3eff Scaled up replica set payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5 from 0 to 1 29m Normal Created pod/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-mc8nv Container created 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5 Created pod: payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-ld47d 29m Normal Pulled pod/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-mc8nv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5 Error creating: pods "payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5f3eff: serviceaccount "payload-processing-e2e-ipp-b-5f3eff" not found 29m Normal Started pod/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-mc8nv Container started 29m Normal AddedInterface pod/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-mc8nv Add eth0 [10.133.1.11/23] from ovn-kubernetes 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b Created pod: payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-mc8nv 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b Error creating: pods "payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5f3eff: serviceaccount "payload-processing-e2e-ipp-b-5f3eff" not found 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-ld47d Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-ld47d Add eth0 [10.133.1.12/23] from ovn-kubernetes 29m Normal Created pod/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-ld47d Container created 29m Normal Started pod/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-ld47d Container started 29m Normal Killing pod/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 Stopping container payload-processing 29m Normal Killing pod/payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c734f5-c79bb986c-nxcxx Stopping container payload-pre-processing 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c173d8-6d47cf995d Error creating: pods "payload-processing-e2e-ipp-a-c173d8-6d47cf995d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c173d8: serviceaccount "payload-processing-e2e-ipp-a-c173d8" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c173d8-6d47cf995d Created pod: payload-processing-e2e-ipp-a-c173d8-6d47cf995d-2bjft 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547 Created pod: payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-zwgd7 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547 Error creating: pods "payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c173d8: serviceaccount "payload-processing-e2e-ipp-a-c173d8" not found 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c173d8 Scaled up replica set payload-processing-e2e-ipp-a-c173d8-6d47cf995d from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c173d8 Scaled up replica set payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547 from 0 to 1 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-zwgd7 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-zwgd7 Container created 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-zwgd7 Add eth0 [10.133.1.14/23] from ovn-kubernetes 29m Normal Started pod/payload-processing-e2e-ipp-a-c173d8-6d47cf995d-2bjft Container started 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-zwgd7 Container started 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c173d8-6d47cf995d-2bjft Add eth0 [10.133.1.13/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-ipp-a-c173d8-6d47cf995d-2bjft Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-processing-e2e-ipp-a-c173d8-6d47cf995d-2bjft Container created 29m Warning Unhealthy pod/payload-processing-e2e-ipp-a-c734f5-699f7df769-c5swh Readiness probe failed: timeout: failed to connect service "10.133.1.5:50052" within 1s: context deadline exceeded 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-5bbf75-7fc4ff59f9-678zw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-processing-e2e-worker-w5-5bbf75-6db9b4cfd7-j4pv4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 Deleted pod: e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h 29m Normal Killing pod/payload-processing-e2e-ipp-a-26cd59-84899b8c48-bgz8c Stopping container payload-processing 29m Normal Killing pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-wlc7h Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-a891ce-openshift-default Scaled down replica set e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 from 4 to 3 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-a891ce-openshift-default New size: 3; reason: All metrics below target 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-26cd59-585984b9f5-z4pwr Stopping container payload-pre-processing 29m Normal SuccessfulCreate replicaset/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95 Created pod: e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 29m Normal AddedLabel gateway/e2e-ipp-b-c9b9a1 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-c9b9a1 29m Normal ScalingReplicaSet deployment/e2e-ipp-b-c9b9a1-openshift-default Scaled up replica set e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95 from 0 to 1 29m Warning Unhealthy pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Readiness probe failed: Get "http://10.133.0.246:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning ProbeError pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Readiness probe error: Get "http://10.133.0.246:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Normal AddedInterface pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Add eth0 [10.133.1.16/23] from ovn-kubernetes 29m Warning ProbeError pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Readiness probe error: Get "http://10.133.0.240:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Warning Unhealthy pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Readiness probe failed: Get "http://10.133.0.240:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-c9b9a1 Scaled up replica set payload-processing-e2e-ipp-b-c9b9a1-5c968dc557 from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557 Created pod: payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-lrnmf 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557 Error creating: pods "payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c9b9a1: serviceaccount "payload-processing-e2e-ipp-b-c9b9a1" not found 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-c9b9a1 Scaled up replica set payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf Created pod: payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-7krhv 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf Error creating: pods "payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c9b9a1: serviceaccount "payload-processing-e2e-ipp-b-c9b9a1" not found 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-7krhv Add eth0 [10.133.1.17/23] from ovn-kubernetes 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d Error creating: pods "payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c734f5: serviceaccount "payload-processing-e2e-ipp-a-c734f5" not found 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c734f5 Scaled up replica set payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5 from 0 to 1 29m Normal Started pod/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-7krhv Container started 29m Normal AddedInterface pod/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-lrnmf Add eth0 [10.133.1.18/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-lrnmf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-bd1c9f-6dbbc4dc79-v5wmk Stopping container payload-pre-processing 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c734f5 Scaled up replica set payload-processing-e2e-ipp-a-c734f5-684c4f6c9d from 0 to 1 29m Normal Created pod/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-lrnmf Container created 29m Normal Created pod/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-7krhv Container created 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d Created pod: payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-xdjpr 29m Normal Started pod/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-lrnmf Container started 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5 Error creating: pods "payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c734f5: serviceaccount "payload-processing-e2e-ipp-a-c734f5" not found 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-7krhv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Killing pod/payload-processing-e2e-ipp-a-bd1c9f-66bb876fcf-qnnw8 Stopping container payload-processing 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5 Created pod: payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-bs8jm 29m Normal Started pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Container started 29m Normal Pulled pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Container created 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-bs8jm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-xdjpr Container started 29m Normal Created pod/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-xdjpr Container created 29m Normal Pulled pod/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-xdjpr Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-xdjpr Add eth0 [10.133.1.20/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-bs8jm Add eth0 [10.133.1.19/23] from ovn-kubernetes 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-bs8jm Container created 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-bs8jm Container started 29m Normal Killing pod/e2e-ipp-a-26cd59-openshift-default-74bbc85d85-kk2sq Stopping container istio-proxy 29m Normal Pulled pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Container created 29m Normal Started pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Container started 29m Warning Unhealthy pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Startup probe failed: Get "http://10.133.0.246:15021/healthz/ready": dial tcp 10.133.0.246:15021: connect: connection refused 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-5f3eff-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning ProbeError pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Startup probe error: Get "http://10.133.0.246:15021/healthz/ready": dial tcp 10.133.0.246:15021: connect: connection refused... 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-5f3eff-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-c734f5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 29m Warning ProbeError pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Startup probe error: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Startup probe failed: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused 29m Normal SuccessfulCreate replicaset/e2e-ipp-a-c734f5-openshift-default-7f99c45bc Created pod: e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w 29m Normal ScalingReplicaSet deployment/e2e-ipp-a-c734f5-openshift-default Scaled up replica set e2e-ipp-a-c734f5-openshift-default-7f99c45bc from 1 to 4 29m Normal Started pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w Container started 29m Normal Pulled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Pulled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w Container created 29m Normal SuccessfulCreate replicaset/e2e-ipp-a-c734f5-openshift-default-7f99c45bc Created pod: e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr 29m Normal Pulled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w Add eth0 [10.133.1.21/23] from ovn-kubernetes 29m Normal Started pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Container started 29m Normal AddedInterface pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Add eth0 [10.132.1.16/23] from ovn-kubernetes 29m Normal AddedInterface pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l Add eth0 [10.134.0.223/23] from ovn-kubernetes 29m Normal Created pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Container created 29m Normal Started pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Container started 29m Normal Created pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l Container created 29m Normal Created pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Container created 29m Normal Pulled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal SuccessfulCreate replicaset/e2e-ipp-a-c734f5-openshift-default-7f99c45bc Created pod: e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l 29m Normal Started pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l Container started 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 Created pod: e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e1d060-openshift-default Scaled up replica set e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65 from 2 to 3 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-e1d060-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Normal Started pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Container started 29m Normal AddedInterface pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Add eth0 [10.134.0.224/23] from ovn-kubernetes 29m Normal Pulled pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Container created 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd Created pod: payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-2bqtm 29m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w Startup probe failed: Get "http://10.133.1.21:15021/healthz/ready": dial tcp 10.133.1.21:15021: connect: connection refused 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-bd1c9f Scaled up replica set payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-bd1c9f Scaled up replica set payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75 from 0 to 1 29m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l Startup probe failed: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75 Error creating: pods "payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-bd1c9f: serviceaccount "payload-processing-e2e-ipp-a-bd1c9f" not found 29m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l Startup probe error: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused... 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75 Created pod: payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-mzfnm 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd Error creating: pods "payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-bd1c9f: serviceaccount "payload-processing-e2e-ipp-a-bd1c9f" not found 29m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w Startup probe error: Get "http://10.133.1.21:15021/healthz/ready": dial tcp 10.133.1.21:15021: connect: connection refused... 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-mzfnm Add eth0 [10.133.1.22/23] from ovn-kubernetes 29m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-h89zs Stopping container istio-proxy 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-2bqtm Add eth0 [10.133.1.23/23] from ovn-kubernetes 29m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Startup probe failed: Get "http://10.132.1.16:15021/healthz/ready": dial tcp 10.132.1.16:15021: connect: connection refused 29m Normal Pulled pod/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-2bqtm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Startup probe error: Get "http://10.132.1.16:15021/healthz/ready": dial tcp 10.132.1.16:15021: connect: connection refused... 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-mzfnm Container created 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-mzfnm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-mzfnm Container started 29m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Startup probe error: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused... 29m Normal Killing pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Stopping container istio-proxy 29m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Startup probe failed: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused 28m Normal Created pod/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-2bqtm Container created 28m Normal AddedLabel gateway/e2e-ipp-b-d7647c Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-d7647c 28m Normal ScalingReplicaSet deployment/e2e-ipp-b-d7647c-openshift-default Scaled up replica set e2e-ipp-b-d7647c-openshift-default-5745b7cd5f from 0 to 1 28m Normal SuccessfulCreate replicaset/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f Created pod: e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt 28m Normal Started pod/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-2bqtm Container started 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c173d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c173d8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal AddedInterface pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Add eth0 [10.132.1.18/23] from ovn-kubernetes 28m Warning Unhealthy pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Startup probe failed: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Startup probe error: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused... 28m Warning ProbeError pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-worker-w5-5bbf75-openshift-default-7cf9cc5c65-9m9rc Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-d7647c Scaled up replica set payload-processing-e2e-ipp-b-d7647c-7457796869 from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-d7647c-7457796869 Created pod: payload-processing-e2e-ipp-b-d7647c-7457796869-bbpbw 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579 Created pod: payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-hs4dc 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579 Error creating: pods "payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d7647c: serviceaccount "payload-processing-e2e-ipp-b-d7647c" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-d7647c Scaled up replica set payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579 from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-d7647c-7457796869 Error creating: pods "payload-processing-e2e-ipp-b-d7647c-7457796869-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d7647c: serviceaccount "payload-processing-e2e-ipp-b-d7647c" not found 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-hs4dc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-hs4dc Container created 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-d7647c-7457796869-bbpbw Add eth0 [10.134.0.225/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-hs4dc Add eth0 [10.134.0.226/23] from ovn-kubernetes 28m Normal Created pod/payload-processing-e2e-ipp-b-d7647c-7457796869-bbpbw Container created 28m Normal Started pod/payload-processing-e2e-ipp-b-d7647c-7457796869-bbpbw Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-b-d7647c-7457796869-bbpbw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-hs4dc Container started 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c734f5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c734f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Killing pod/payload-processing-e2e-ipp-b-5f3eff-65c86b7b8b-mc8nv Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-b-5f3eff-7db97588f5-ld47d Stopping container payload-pre-processing 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-5f3eff-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-5f3eff-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c173d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c173d8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal ScalingReplicaSet deployment/e2e-ipp-b-a84fcb-openshift-default Scaled up replica set e2e-ipp-b-a84fcb-openshift-default-5778f8f47c from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-b-a84fcb Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-a84fcb 28m Normal SuccessfulCreate replicaset/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c Created pod: e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb 28m Normal AddedInterface pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Add eth0 [10.133.1.24/23] from ovn-kubernetes 28m Normal Created pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Container created 28m Normal Started pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Container started 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-a84fcb Scaled up replica set payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859 from 0 to 1 28m Normal Created pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Container created 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-a84fcb-bdbd55747 Error creating: pods "payload-processing-e2e-ipp-b-a84fcb-bdbd55747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a84fcb: serviceaccount "payload-processing-e2e-ipp-b-a84fcb" not found 28m Normal Pulled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-c734f5-openshift-default Scaled up replica set e2e-ipp-a-c734f5-openshift-default-7f99c45bc from 4 to 7 28m Normal AddedInterface pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Add eth0 [10.132.1.19/23] from ovn-kubernetes 28m Normal Started pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Container started 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-c734f5-openshift-default New size: 7; reason: 28m Normal Pulled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Add eth0 [10.134.0.227/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-c734f5-openshift-default-7f99c45bc Created pod: e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-a84fcb Scaled up replica set payload-processing-e2e-ipp-b-a84fcb-bdbd55747 from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859 Created pod: payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-5685f 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859 Error creating: pods "payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a84fcb: serviceaccount "payload-processing-e2e-ipp-b-a84fcb" not found 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-c734f5-openshift-default-7f99c45bc Created pod: e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-c734f5-openshift-default-7f99c45bc Created pod: e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc 28m Normal AddedInterface pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Add eth0 [10.133.1.25/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Killing pod/e2e-ipp-b-5f3eff-openshift-default-78b5fb4875-7qq8v Stopping container istio-proxy 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-a84fcb-bdbd55747 Created pod: payload-processing-e2e-ipp-b-a84fcb-bdbd55747-zkl6p 28m Normal Started pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Container started 28m Normal Created pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Container created 28m Normal Created pod/payload-processing-e2e-ipp-b-a84fcb-bdbd55747-zkl6p Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-5685f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-5685f Container started 28m Normal Started pod/payload-processing-e2e-ipp-b-a84fcb-bdbd55747-zkl6p Container started 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-5685f Container created 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-5685f Add eth0 [10.134.0.228/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-a84fcb-bdbd55747-zkl6p Add eth0 [10.134.0.229/23] from ovn-kubernetes 28m Normal Pulled pod/payload-processing-e2e-ipp-b-a84fcb-bdbd55747-zkl6p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Pulled pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Container created 28m Normal Started pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Container started 28m Normal Killing pod/payload-pre-processing-e2e-ipp-b-c9b9a1-79688949cf-7krhv Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-ipp-b-c9b9a1-5c968dc557-lrnmf Stopping container payload-processing 28m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Startup probe failed: Get "http://10.133.1.25:15021/healthz/ready": dial tcp 10.133.1.25:15021: connect: connection refused 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-bd1c9f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Startup probe error: Get "http://10.132.1.19:15021/healthz/ready": dial tcp 10.132.1.19:15021: connect: connection refused... 28m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Startup probe error: Get "http://10.133.1.25:15021/healthz/ready": dial tcp 10.133.1.25:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Startup probe failed: Get "http://10.132.1.19:15021/healthz/ready": dial tcp 10.132.1.19:15021: connect: connection refused 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-bd1c9f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Startup probe failed: Get "http://10.134.0.227:15021/healthz/ready": dial tcp 10.134.0.227:15021: connect: connection refused 28m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Startup probe error: Get "http://10.134.0.227:15021/healthz/ready": dial tcp 10.134.0.227:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Startup probe failed: Get "http://10.133.1.7:15021/healthz/ready": dial tcp 10.133.1.7:15021: connect: connection refused 28m Warning ProbeError pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Startup probe error: Get "http://10.133.1.7:15021/healthz/ready": dial tcp 10.133.1.7:15021: connect: connection refused... 28m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 Stopping container payload-processing 28m Normal Killing pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Stopping container istio-proxy 28m Normal Killing pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Stopping container istio-proxy 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c Created pod: payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-b9svq 28m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd Error creating: pods "payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c9b9a1: serviceaccount "payload-processing-e2e-ipp-b-c9b9a1" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-c9b9a1 Scaled up replica set payload-processing-e2e-ipp-b-c9b9a1-f45479ffd from 0 to 1 28m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-qdqkv Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-c9b9a1 Scaled up replica set payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c Error creating: pods "payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-c9b9a1: serviceaccount "payload-processing-e2e-ipp-b-c9b9a1" not found 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-b9svq Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-b9svq Add eth0 [10.133.1.28/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd Created pod: payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-mqgll 28m Normal Pulled pod/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-mqgll Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-b9svq Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-b9svq Container started 28m Warning ProbeError pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal Created pod/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-mqgll Container created 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-mqgll Add eth0 [10.133.1.29/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-mqgll Container started 28m Warning Unhealthy pod/e2e-shared-a-w0-fd1c91-openshift-default-7f584ff8ff-6hdmk Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Readiness probe failed: Get "http://10.133.1.16:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Warning ProbeError pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Readiness probe error: Get "http://10.133.1.16:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-fd1c91-5fb9c8fcf9-xvrkd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll Stopping container payload-pre-processing 28m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-fd1c91-84dcbc9465-5r662 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/payload-processing-e2e-ipp-b-d7647c-7457796869-bbpbw Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-b-d7647c-5cccb98579-hs4dc Stopping container payload-pre-processing 28m Normal Started pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Container started 28m Normal Pulled pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Container created 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-bd1c9f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-bd1c9f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning ProbeError pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Startup probe error: Get "http://10.133.1.16:15021/healthz/ready": dial tcp 10.133.1.16:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Startup probe failed: Get "http://10.133.1.16:15021/healthz/ready": dial tcp 10.133.1.16:15021: connect: connection refused 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-c9b9a1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-c9b9a1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Normal Killing pod/payload-processing-e2e-ipp-b-a84fcb-bdbd55747-zkl6p Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-ipp-b-a84fcb-6f899ff859-5685f Stopping container payload-pre-processing 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-d7647c Scaled up replica set payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f Created pod: payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-qvj7p 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-d7647c-66bbb74874 Created pod: payload-processing-e2e-ipp-b-d7647c-66bbb74874-4dsdn 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-d7647c-66bbb74874 Error creating: pods "payload-processing-e2e-ipp-b-d7647c-66bbb74874-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d7647c: serviceaccount "payload-processing-e2e-ipp-b-d7647c" not found 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f Error creating: pods "payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d7647c: serviceaccount "payload-processing-e2e-ipp-b-d7647c" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-d7647c Scaled up replica set payload-processing-e2e-ipp-b-d7647c-66bbb74874 from 0 to 1 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-qvj7p Add eth0 [10.133.1.32/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-d7647c-66bbb74874-4dsdn Add eth0 [10.133.1.31/23] from ovn-kubernetes 27m Normal Killing pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-qqcjr Stopping container istio-proxy 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-qvj7p Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-qvj7p Container created 27m Normal Killing pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Stopping container istio-proxy 27m Normal Created pod/payload-processing-e2e-ipp-b-d7647c-66bbb74874-4dsdn Container created 27m Normal Pulled pod/payload-processing-e2e-ipp-b-d7647c-66bbb74874-4dsdn Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Killing pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-774z4 Stopping container istio-proxy 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-qvj7p Container started 27m Normal Started pod/payload-processing-e2e-ipp-b-d7647c-66bbb74874-4dsdn Container started 27m Warning Unhealthy pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-shared-b-w0-f5c1ab-openshift-default-5749cfc96f-shwc2 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-f5c1ab-54c4d89f99-b7fr8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-f5c1ab-68b5bdf4d7-jwcll Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz Stopping container payload-pre-processing 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9 Created pod: payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-hmqdf 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-a84fcb Scaled up replica set payload-processing-e2e-ipp-b-a84fcb-5465fc5b64 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64 Created pod: payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-v2fc5 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-a84fcb Scaled up replica set payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9 from 0 to 1 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-hmqdf Add eth0 [10.134.0.231/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64 Error creating: pods "payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a84fcb: serviceaccount "payload-processing-e2e-ipp-b-a84fcb" not found 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9 Error creating: pods "payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a84fcb: serviceaccount "payload-processing-e2e-ipp-b-a84fcb" not found 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-v2fc5 Add eth0 [10.133.1.34/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-v2fc5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-hmqdf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-hmqdf Container created 27m Normal Created pod/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-v2fc5 Container created 27m Normal Started pod/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-v2fc5 Container started 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-hmqdf Container started 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-c9b9a1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-c9b9a1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning ProbeError pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Readiness probe error: Get "http://10.132.1.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 27m Warning Unhealthy pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Readiness probe failed: Get "http://10.132.1.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 27m Warning ProbeError pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Readiness probe error: Get "http://10.133.1.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 27m Normal Created pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Container created 27m Warning Unhealthy pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Readiness probe failed: Get "http://10.133.1.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 27m Normal Pulled pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Container started 27m Normal Pulled pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-435728-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 27m Normal Created pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Container created 27m Warning Unhealthy pod/payload-processing-e2e-worker-w0-23843f-66cb64b99f-v9xjq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-23843f-6dcc5f75c5-jr2rz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-435728-openshift-default Scaled up replica set e2e-shared-b-w1-435728-openshift-default-5f778d7cf7 from 2 to 3 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7 Created pod: e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 27m Normal Started pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Container started 27m Normal Pulled pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 Add eth0 [10.134.0.232/23] from ovn-kubernetes 27m Normal Started pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 Container started 27m Normal Created pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 Container created 27m Warning Unhealthy pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Startup probe failed: Get "http://10.132.1.18:15021/healthz/ready": dial tcp 10.132.1.18:15021: connect: connection refused 27m Warning ProbeError pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Startup probe error: Get "http://10.132.1.18:15021/healthz/ready": dial tcp 10.132.1.18:15021: connect: connection refused... 27m Warning ProbeError pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Startup probe error: Get "http://10.133.1.24:15021/healthz/ready": dial tcp 10.133.1.24:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Startup probe failed: Get "http://10.133.1.24:15021/healthz/ready": dial tcp 10.133.1.24:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 Startup probe failed: Get "http://10.134.0.232:15021/healthz/ready": dial tcp 10.134.0.232:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 Startup probe error: Get "http://10.134.0.232:15021/healthz/ready": dial tcp 10.134.0.232:15021: connect: connection refused... 27m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c734f5-786f585bf5-bs8jm Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-ipp-a-c734f5-684c4f6c9d-xdjpr Stopping container payload-processing 27m Normal Killing pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Stopping container istio-proxy 27m Normal Killing pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fhr8w Stopping container istio-proxy 27m Normal Killing pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Stopping container istio-proxy 27m Normal Killing pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-5552l Stopping container istio-proxy 27m Normal Killing pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Stopping container istio-proxy 27m Normal Killing pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Stopping container istio-proxy 27m Normal Killing pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Stopping container istio-proxy 26m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-tjfkw Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-wvnqr Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-fdk2w Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-nr5p8 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning ProbeError pod/e2e-ipp-a-c734f5-openshift-default-7f99c45bc-2zgsc Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a84fcb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a84fcb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/payload-processing-e2e-ipp-a-bd1c9f-65b6b696bd-2bqtm Stopping container payload-processing 26m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-a-bd1c9f Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-a-bd1c9f: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-a-bd1c9f": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-a-bd1c9f, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: bbb7f7ba-3eee-4ba1-8f0f-f15efd488dd3, UID in object meta: 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-bd1c9f-745586fb75-mzfnm Stopping container payload-pre-processing 26m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-b-d7647c Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-d7647c: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-d7647c": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-d7647c, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: d33249c0-9a96-48da-a04f-7d63cb86a87b, UID in object meta: 26m Normal Killing pod/payload-processing-e2e-ipp-b-d7647c-66bbb74874-4dsdn Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-d7647c-85d9ccfd7f-qvj7p Stopping container payload-pre-processing 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-d7647c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-d7647c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Stopping container istio-proxy 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a84fcb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-bzzjd Stopping container istio-proxy 26m Normal Killing pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Stopping container istio-proxy 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a84fcb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning ProbeError pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-worker-w0-23843f-openshift-default-54cf54885-lb2gv Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning Unhealthy pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-ipp-a-bd1c9f-openshift-default-74d884b45c-7dhw6 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal Killing pod/e2e-ipp-b-d7647c-openshift-default-5745b7cd5f-xw6gt Stopping container istio-proxy 26m Normal AddedLabel gateway/e2e-ipp-a-f28d48 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f28d48 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-f28d48-openshift-default-c89647f6c Created pod: e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-f28d48-openshift-default Scaled up replica set e2e-ipp-a-f28d48-openshift-default-c89647f6c from 0 to 1 26m Normal AddedInterface pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Add eth0 [10.133.1.43/23] from ovn-kubernetes 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75 Created pod: payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-6xd4b 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f28d48-7745df5fc Created pod: payload-processing-e2e-ipp-a-f28d48-7745df5fc-xzcb6 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f28d48 Scaled up replica set payload-processing-e2e-ipp-a-f28d48-7745df5fc from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f28d48-7745df5fc Error creating: pods "payload-processing-e2e-ipp-a-f28d48-7745df5fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f28d48: serviceaccount "payload-processing-e2e-ipp-a-f28d48" not found 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75 Error creating: pods "payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f28d48: serviceaccount "payload-processing-e2e-ipp-a-f28d48" not found 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f28d48 Scaled up replica set payload-pre-processing-e2e-ipp-a-f28d48-74946ff75 from 0 to 1 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-6xd4b Container created 26m Normal Created pod/payload-processing-e2e-ipp-a-f28d48-7745df5fc-xzcb6 Container created 26m Normal Pulled pod/payload-processing-e2e-ipp-a-f28d48-7745df5fc-xzcb6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-6xd4b Add eth0 [10.134.0.233/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-6xd4b Container started 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f28d48-7745df5fc-xzcb6 Add eth0 [10.134.0.234/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-6xd4b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Started pod/payload-processing-e2e-ipp-a-f28d48-7745df5fc-xzcb6 Container started 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-a891ce-openshift-default Scaled up replica set e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 from 3 to 4 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-a891ce-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65 Created pod: e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf 26m Normal Pulled pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Container started 26m Normal AddedInterface pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Add eth0 [10.132.1.26/23] from ovn-kubernetes 26m Normal Created pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Container created 26m Normal Killing pod/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd Stopping container payload-pre-processing 26m Warning Unhealthy pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Startup probe failed: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Startup probe error: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused... 26m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-ae2e60-openshift-default New size: 9; reason: All metrics below target 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ae2e60-openshift-default Scaled down replica set e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 from 10 to 9 26m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47 Deleted pod: e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-w8jm8 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-a84fcb-db98dd6f9-hmqdf Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-b-a84fcb-5465fc5b64-v2fc5 Stopping container payload-processing 26m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Normal Killing pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Stopping container istio-proxy 26m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-v9ds8 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal Killing pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Stopping container istio-proxy 26m Normal Killing pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Stopping container istio-proxy 26m Normal Killing pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Stopping container istio-proxy 26m Normal Killing pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Stopping container istio-proxy 26m Normal Killing pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Stopping container istio-proxy 26m Normal Killing pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Stopping container istio-proxy 26m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-86g8t Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-e618e1-5f6cdf8759-qwvgd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-e618e1-7cddf5f95c-xd7c4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-bxld2 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-hltgr Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-kx8c9 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-7flph Readiness probe failed: HTTP probe failed with statuscode: 503 26m Normal Killing pod/e2e-ipp-b-a84fcb-openshift-default-5778f8f47c-l4tjb Stopping container istio-proxy 26m Warning ProbeError pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Readiness probe error: Get "http://10.132.0.251:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Warning Unhealthy pod/e2e-shared-a-w4-e618e1-openshift-default-6448cf6987-24xrh Readiness probe failed: Get "http://10.132.0.251:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp Stopping container payload-processing 25m Normal Killing pod/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-ipp-a-f28d48-7745df5fc-xzcb6 Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f28d48-74946ff75-6xd4b Stopping container payload-pre-processing 25m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-t8wfk Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-kd7l7 Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4krfl Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Stopping container istio-proxy 25m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-mlkk8 Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-shared-b-w4-d9c048-openshift-default-5dbbb74c57-4pmnf Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f28d48 Scaled up replica set payload-processing-e2e-ipp-a-f28d48-6fc75df778 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f28d48 Scaled up replica set payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f28d48-6fc75df778 Created pod: payload-processing-e2e-ipp-a-f28d48-6fc75df778-4x22w 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f28d48-6fc75df778 Error creating: pods "payload-processing-e2e-ipp-a-f28d48-6fc75df778-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f28d48: serviceaccount "payload-processing-e2e-ipp-a-f28d48" not found 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5 Error creating: pods "payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f28d48: serviceaccount "payload-processing-e2e-ipp-a-f28d48" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5 Created pod: payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-8jbqq 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-8jbqq Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-8jbqq Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f28d48-6fc75df778-4x22w Add eth0 [10.133.1.47/23] from ovn-kubernetes 25m Normal Started pod/payload-processing-e2e-ipp-a-f28d48-6fc75df778-4x22w Container started 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-8jbqq Container started 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-8jbqq Add eth0 [10.133.1.46/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-a-f28d48-6fc75df778-4x22w Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-a-f28d48-6fc75df778-4x22w Container created 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-d9c048-5459f7f8dc-rhrkn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-a891ce-5b8df746d7-qj9jj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-a891ce-f478d8c55-tz4rs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-d9c048-79ff7865f5-qq5qp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-gbjbd Stopping container istio-proxy 25m Normal Killing pod/payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv Stopping container payload-processing 25m Normal Killing pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Stopping container istio-proxy 25m Normal Killing pod/payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt Stopping container payload-pre-processing 25m Normal Killing pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Stopping container istio-proxy 25m Warning ProbeError pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-cqnc6 Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-rcvzs Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning ProbeError pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-a-w6-a891ce-openshift-default-5645c95f65-s5hlf Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-r5j99 Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Stopping container istio-proxy 25m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-4bac5a-78664677cc-6shsd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-4bac5a-656d5db6dc-6mbl5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning ProbeError pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-a-w2-4bac5a-openshift-default-598ccc8647-dmjwc Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk Stopping container payload-processing 25m Normal SuccessfulCreate replicaset/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855 Created pod: e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd 25m Normal AddedLabel gateway/e2e-ipp-b-7fe845 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-7fe845 25m Normal ScalingReplicaSet deployment/e2e-ipp-b-7fe845-openshift-default Scaled up replica set e2e-ipp-b-7fe845-openshift-default-5fdf5fd855 from 0 to 1 25m Normal AddedInterface pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Add eth0 [10.133.1.49/23] from ovn-kubernetes 25m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 Stopping container payload-processing 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-f28d48-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-f28d48-openshift-default-c89647f6c Created pod: e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-f28d48-openshift-default-c89647f6c Created pod: e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-f28d48-openshift-default-c89647f6c Created pod: e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-f28d48-openshift-default Scaled up replica set e2e-ipp-a-f28d48-openshift-default-c89647f6c from 1 to 4 25m Normal Started pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Container started 25m Warning Unhealthy pod/payload-processing-e2e-worker-w4-996c32-5b66846755-x5jkv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Started pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Container started 25m Normal Started pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Container started 25m Normal Created pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Container created 25m Normal Pulled pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Add eth0 [10.134.0.236/23] from ovn-kubernetes 25m Normal Created pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Container created 25m Normal Pulled pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Add eth0 [10.132.1.32/23] from ovn-kubernetes 25m Normal Created pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Container created 25m Normal Pulled pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Add eth0 [10.133.1.50/23] from ovn-kubernetes 25m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-996c32-7585565cff-4bwmt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9 Error creating: pods "payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7fe845: serviceaccount "payload-processing-e2e-ipp-b-7fe845" not found 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-7fe845 Scaled up replica set payload-processing-e2e-ipp-b-7fe845-6db886499c from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-7fe845-6db886499c Created pod: payload-processing-e2e-ipp-b-7fe845-6db886499c-vfjqn 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-7fe845-6db886499c Error creating: pods "payload-processing-e2e-ipp-b-7fe845-6db886499c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7fe845: serviceaccount "payload-processing-e2e-ipp-b-7fe845" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9 Created pod: payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-ft6fm 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-7fe845 Scaled up replica set payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9 from 0 to 1 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-ft6fm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Warning ProbeError pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Readiness probe error: Get "http://10.133.1.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Normal Created pod/payload-processing-e2e-ipp-b-7fe845-6db886499c-vfjqn Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-b-7fe845-6db886499c-vfjqn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Readiness probe failed: Get "http://10.133.1.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Normal Started pod/payload-processing-e2e-ipp-b-7fe845-6db886499c-vfjqn Container started 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-ft6fm Add eth0 [10.134.0.237/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-ft6fm Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-ft6fm Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-7fe845-6db886499c-vfjqn Add eth0 [10.134.0.238/23] from ovn-kubernetes 25m Normal Started pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Container started 25m Normal Pulled pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Container created 25m Warning ProbeError pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Startup probe error: Get "http://10.134.0.236:15021/healthz/ready": dial tcp 10.134.0.236:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Startup probe failed: Get "http://10.134.0.236:15021/healthz/ready": dial tcp 10.134.0.236:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Startup probe error: Get "http://10.132.1.32:15021/healthz/ready": dial tcp 10.132.1.32:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Startup probe failed: Get "http://10.132.1.32:15021/healthz/ready": dial tcp 10.132.1.32:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Startup probe error: Get "http://10.133.1.50:15021/healthz/ready": dial tcp 10.133.1.50:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Startup probe failed: Get "http://10.133.1.50:15021/healthz/ready": dial tcp 10.133.1.50:15021: connect: connection refused 25m Normal Killing pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-gzmxl Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Stopping container istio-proxy 25m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Startup probe failed: Get "http://10.133.1.43:15021/healthz/ready": dial tcp 10.133.1.43:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Startup probe error: Get "http://10.133.1.43:15021/healthz/ready": dial tcp 10.133.1.43:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-sb6nr Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-x5jh2 Stopping container istio-proxy 25m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning ProbeError pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-6zgng Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-pbzds Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-g5qg8 Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-4dc4b Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-bqfpk Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-s75qz Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning ProbeError pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-v7d6l Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-b-w2-ae2e60-openshift-default-5787499c47-jnkxm Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-b-w6-e1d060-openshift-default-75d9d4df65-cj6sd Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f28d48-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f28d48-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-e1d060-7dfc4c5fc5-l25gk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Stopping container istio-proxy 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-e1d060-78dc6c4c7f-fv2mr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-mrrfb Stopping container istio-proxy 25m Warning Unhealthy pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Stopping container payload-processing 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-ae2e60-68646c459f-7mfvj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-ae2e60-67c555dfb9-t8bs7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning ProbeError pod/e2e-worker-w4-996c32-openshift-default-58dc84fb9-g9hvs Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal Killing pod/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-b-7fe845-6db886499c-vfjqn Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-b-7fe845-79c665fbc9-ft6fm Stopping container payload-pre-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c173d8-7dbb76c547-zwgd7 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-a-c173d8-6d47cf995d-2bjft Stopping container payload-processing 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-7fe845 Scaled up replica set payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7 Created pod: payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-4j9n2 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-7fe845 Scaled up replica set payload-processing-e2e-ipp-b-7fe845-7c7f967f6b from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b Created pod: payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-dqxhs 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b Error creating: pods "payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7fe845: serviceaccount "payload-processing-e2e-ipp-b-7fe845" not found 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7 Error creating: pods "payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7fe845: serviceaccount "payload-processing-e2e-ipp-b-7fe845" not found 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-4j9n2 Container started 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-4j9n2 Add eth0 [10.134.0.240/23] from ovn-kubernetes 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-dqxhs Add eth0 [10.134.0.241/23] from ovn-kubernetes 24m Normal Pulled pod/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-dqxhs Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-4j9n2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-dqxhs Container created 24m Normal Started pod/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-dqxhs Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-4j9n2 Container created 24m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-vzbln Stopping container istio-proxy 24m Normal Killing pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Stopping container istio-proxy 24m Warning Unhealthy pod/payload-processing-e2e-worker-w2-aeb292-7b88bf8765-wcrb9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-8730d5-569cd886cf-shqb5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning ProbeError pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-worker-w2-aeb292-openshift-default-58cb568b9f-pnhwk Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-aeb292-5fbf46df9-w8bqt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-processing-e2e-worker-w6-8730d5-7f459f7c9c-ss9sq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Pulled pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Container created 24m Normal Started pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Container started 24m Warning ProbeError pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Startup probe error: Get "http://10.133.1.49:15021/healthz/ready": dial tcp 10.133.1.49:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Startup probe failed: Get "http://10.133.1.49:15021/healthz/ready": dial tcp 10.133.1.49:15021: connect: connection refused 24m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-ltmpt Stopping container istio-proxy 24m Normal Killing pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Stopping container istio-proxy 24m Warning ProbeError pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-worker-w6-8730d5-openshift-default-59d8dd4587-9jcjm Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal Killing pod/payload-processing-e2e-ipp-a-f28d48-6fc75df778-4x22w Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f28d48-5bbcbb6f5-8jbqq Stopping container payload-pre-processing 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-7fe845-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-7fe845-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal Killing pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-lqwbc Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Stopping container istio-proxy 23m Warning Unhealthy pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Stopping container istio-proxy 23m Warning ProbeError pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-9sjnc Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-kgg4j Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning ProbeError pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-a-f28d48-openshift-default-c89647f6c-8gsbc Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-a-c173d8-openshift-default-6945698f95-6fj9b Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-7fe845-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-7fe845-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/payload-processing-e2e-ipp-b-7fe845-7c7f967f6b-dqxhs Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-7fe845-754c6876d7-4j9n2 Stopping container payload-pre-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-c9b9a1-666b4798c-b9svq Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-c9b9a1-f45479ffd-mqgll Stopping container payload-processing 23m Normal Killing pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Stopping container istio-proxy 23m Normal Killing pod/e2e-ipp-b-c9b9a1-openshift-default-bb66f5c95-wgvv8 Stopping container istio-proxy 23m Warning ProbeError pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-b-7fe845-openshift-default-5fdf5fd855-br6xd Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w Stopping container payload-processing 23m Normal Killing pod/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 Stopping container payload-pre-processing 23m Normal Killing pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-wxqll Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-a-w1-cd9729-openshift-default-6cdbfbb869-rsj59 Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-5g7p8 Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-a-w3-c55cf1-openshift-default-5c956b6f87-m5f25 Stopping container istio-proxy 23m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-cd9729-774576d6f5-g8n6w Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-cd9729-77b49dfbbc-m7jw6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 Stopping container payload-pre-processing 23m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-c55cf1-676466dd6f-s8jbg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/payload-processing-e2e-shared-b-w3-ae4ce2-75f79f6df7-h7nqs Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd Stopping container payload-pre-processing 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-c55cf1-6ff96ddc6c-np8m2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-vbzt5 Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-2fp29 Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w1-435728-openshift-default-5f778d7cf7-gz4q5 Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-sf9v4 Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w3-ae4ce2-openshift-default-658c56f79f-86d45 Stopping container istio-proxy 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-ae4ce2-5f87bc98dc-dzhpd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-435728-7b9576b5ff-xrkt7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g Stopping container payload-processing 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-435728-5fd9766559-wqr48 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-worker-w1-8b083a Failed to update endpoint openshift-ingress/payload-processing-e2e-worker-w1-8b083a: Operation cannot be fulfilled on endpoints "payload-processing-e2e-worker-w1-8b083a": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-worker-w1-8b083a, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b615355a-060d-41a3-9744-44c46b182d35, UID in object meta: 22m Normal Killing pod/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz Stopping container payload-pre-processing 22m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-j7bc4 Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Stopping container istio-proxy 22m Warning Unhealthy pod/payload-processing-e2e-worker-w1-8b083a-67f5ddc8b5-pcl5g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-8b083a-76688799f5-29hdz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning ProbeError pod/e2e-worker-w1-8b083a-openshift-default-f965c5bc5-zmszd Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bn868 Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-ph4dr Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w3-23814d-openshift-default-775b6897bc-bgvwt Stopping container istio-proxy 22m Warning Unhealthy pod/payload-processing-e2e-worker-w3-23814d-6cb8b9dc85-scg98 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-23814d-75cfff9b75-qwtzz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-qqmwf Container created 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Add eth0 [10.133.1.64/23] from ovn-kubernetes 21m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-55znv Container started 21m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-55znv Container created 21m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-55znv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75d7c785f from 4 to 8 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-55znv Add eth0 [10.133.1.65/23] from ovn-kubernetes 21m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-qqmwf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-qqmwf Add eth0 [10.132.1.37/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-55znv 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-fhccm Add eth0 [10.132.1.38/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-qqmwf 21m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-fhccm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-vkszn 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-fhccm 21m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Container started 21m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Container created 21m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-fhccm Container created 21m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-fhccm Container started 21m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-qqmwf Container started 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-fhccm Startup probe error: Get "http://10.132.1.38:15021/healthz/ready": dial tcp 10.132.1.38:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-fhccm Startup probe failed: Get "http://10.132.1.38:15021/healthz/ready": dial tcp 10.132.1.38:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-qqmwf Startup probe failed: Get "http://10.132.1.37:15021/healthz/ready": dial tcp 10.132.1.37:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Startup probe error: Get "http://10.133.1.64:15021/healthz/ready": dial tcp 10.133.1.64:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Startup probe failed: Get "http://10.133.1.64:15021/healthz/ready": dial tcp 10.133.1.64:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-qqmwf Startup probe error: Get "http://10.132.1.37:15021/healthz/ready": dial tcp 10.132.1.37:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-55znv Startup probe failed: Get "http://10.133.1.65:15021/healthz/ready": dial tcp 10.133.1.65:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-55znv Startup probe error: Get "http://10.133.1.65:15021/healthz/ready": dial tcp 10.133.1.65:15021: connect: connection refused... 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-vj5qn 21m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-6ncmf Container created 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-6ncmf 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-6ncmf Add eth0 [10.134.0.249/23] from ovn-kubernetes 21m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-6ncmf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75d7c785f from 8 to 10 21m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Add eth0 [10.133.1.66/23] from ovn-kubernetes 21m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-6ncmf Container started 21m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Container created 21m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Container started 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6ncmf Startup probe failed: Get "http://10.134.0.249:15021/healthz/ready": dial tcp 10.134.0.249:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Startup probe error: Get "http://10.133.1.66:15021/healthz/ready": dial tcp 10.133.1.66:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6ncmf Startup probe error: Get "http://10.134.0.249:15021/healthz/ready": dial tcp 10.134.0.249:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Startup probe failed: Get "http://10.133.1.66:15021/healthz/ready": dial tcp 10.133.1.66:15021: connect: connection refused 21m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-bhgqx Stopping container discovery 21m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 21m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 21m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-bhgqx 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Readiness probe failed: Get "http://10.133.1.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Readiness probe error: Get "http://10.133.1.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 20m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Readiness probe error: Get "http://10.132.0.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-w7w5h Readiness probe failed: Get "http://10.132.0.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 19m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Readiness probe error: Get "http://10.133.1.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Readiness probe failed: Get "http://10.133.1.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 19m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-2q74f 19m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-2q74f Stopping container discovery 19m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 19m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-6ncmf 11m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-vj5qn Stopping container istio-proxy 11m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 11m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75d7c785f from 10 to 4 11m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-qqmwf Stopping container istio-proxy 11m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-6k6cd Stopping container istio-proxy 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-vj5qn 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-qqmwf 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-6k6cd 11m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-55znv Stopping container istio-proxy 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-55znv 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-vkszn 11m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-6ncmf Stopping container istio-proxy 11m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-vkszn Stopping container istio-proxy 11m Normal AddedLabel gateway/e2e-worker-single-main-844db7 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-844db7 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-844db7-openshift-default Scaled up replica set e2e-worker-single-main-844db7-openshift-default-54d4f5f85 from 0 to 1 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-844db7-openshift-default-54d4f5f85 Created pod: e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w 11m Normal AddedInterface pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Add eth0 [10.133.1.67/23] from ovn-kubernetes 11m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-844db7-d587bc4cf Error creating: pods "payload-pre-processing-e2e-worker-single-main-844db7-d587bc4cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-844db7: serviceaccount "payload-processing-e2e-worker-single-main-844db7" not found 11m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-844db7 Scaled up replica set payload-pre-processing-e2e-worker-single-main-844db7-d587bc4cf from 0 to 1 11m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-844db7-6f7649b585 Error creating: pods "payload-processing-e2e-worker-single-main-844db7-6f7649b585-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-844db7: serviceaccount "payload-processing-e2e-worker-single-main-844db7" not found 11m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-844db7-6f7649b585 Created pod: payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw 11m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-844db7-d587bc4cf Created pod: payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg 11m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-844db7 Scaled up replica set payload-processing-e2e-worker-single-main-844db7-6f7649b585 from 0 to 1 11m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg Add eth0 [10.133.1.69/23] from ovn-kubernetes 11m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal Created pod/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw Container created 11m Normal Started pod/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw Container started 11m Normal Started pod/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg Container started 11m Normal Created pod/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg Container created 11m Normal Pulled pod/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw Add eth0 [10.133.1.70/23] from ovn-kubernetes 11m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75d7c785f from 4 to 3 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75d7c785f Deleted pod: maas-default-gateway-openshift-default-75d7c785f-fhccm 11m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 11m Normal Killing pod/maas-default-gateway-openshift-default-75d7c785f-fhccm Stopping container istio-proxy 11m Normal Created pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Container created 11m Normal Pulled pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Started pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Container started 11m Warning Unhealthy pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Startup probe failed: Get "http://10.133.1.67:15021/healthz/ready": dial tcp 10.133.1.67:15021: connect: connection refused 11m Warning ProbeError pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Startup probe error: Get "http://10.133.1.67:15021/healthz/ready": dial tcp 10.133.1.67:15021: connect: connection refused... 11m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Container started 11m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Add eth0 [10.133.1.72/23] from ovn-kubernetes 11m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Container created 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-6rwmb 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-68qbd 11m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Container started 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-qgfcs 11m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75d7c785f from 3 to 6 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Add eth0 [10.132.1.40/23] from ovn-kubernetes 11m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Add eth0 [10.133.1.71/23] from ovn-kubernetes 11m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Container created 11m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Container started 11m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Container created 11m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Startup probe error: Get "http://10.132.1.40:15021/healthz/ready": dial tcp 10.132.1.40:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Startup probe failed: Get "http://10.132.1.40:15021/healthz/ready": dial tcp 10.132.1.40:15021: connect: connection refused 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Startup probe failed: Get "http://10.133.1.71:15021/healthz/ready": dial tcp 10.133.1.71:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Startup probe error: Get "http://10.133.1.72:15021/healthz/ready": dial tcp 10.133.1.72:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Startup probe failed: Get "http://10.133.1.72:15021/healthz/ready": dial tcp 10.133.1.72:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Startup probe error: Get "http://10.133.1.71:15021/healthz/ready": dial tcp 10.133.1.71:15021: connect: connection refused... 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-f92ld 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Add eth0 [10.133.1.74/23] from ovn-kubernetes 11m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Container started 11m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Container started 11m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Container created 11m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-lwvpn Container started 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-lwvpn 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-pdvwj 11m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75d7c785f Created pod: maas-default-gateway-openshift-default-75d7c785f-t27sn 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Add eth0 [10.132.1.41/23] from ovn-kubernetes 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-lwvpn Add eth0 [10.133.1.73/23] from ovn-kubernetes 11m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-lwvpn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-lwvpn Container created 11m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: All metrics below target 11m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Container created 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-75d7c785f-pdvwj Add eth0 [10.134.1.9/23] from ovn-kubernetes 11m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-pdvwj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Started pod/maas-default-gateway-openshift-default-75d7c785f-pdvwj Container started 11m Normal Pulled pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Created pod/maas-default-gateway-openshift-default-75d7c785f-pdvwj Container created 11m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75d7c785f from 6 to 10 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-844db7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-844db7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-pdvwj Startup probe error: Get "http://10.134.1.9:15021/healthz/ready": dial tcp 10.134.1.9:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-pdvwj Startup probe failed: Get "http://10.134.1.9:15021/healthz/ready": dial tcp 10.134.1.9:15021: connect: connection refused 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-lwvpn Startup probe failed: Get "http://10.133.1.73:15021/healthz/ready": dial tcp 10.133.1.73:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Startup probe error: Get "http://10.132.1.41:15021/healthz/ready": dial tcp 10.132.1.41:15021: connect: connection refused... 11m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-lwvpn Startup probe error: Get "http://10.133.1.73:15021/healthz/ready": dial tcp 10.133.1.73:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Startup probe failed: Get "http://10.132.1.41:15021/healthz/ready": dial tcp 10.132.1.41:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Startup probe error: Get "http://10.133.1.74:15021/healthz/ready": dial tcp 10.133.1.74:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Startup probe failed: Get "http://10.133.1.74:15021/healthz/ready": dial tcp 10.133.1.74:15021: connect: connection refused 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-844db7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-844db7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 10m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg Stopping container payload-pre-processing 10m Normal Killing pod/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw Stopping container payload-processing 10m Normal Killing pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Stopping container istio-proxy 10m Warning ProbeError pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Readiness probe error: HTTP probe failed with statuscode: 503... 10m Warning Unhealthy pod/e2e-worker-single-main-844db7-openshift-default-54d4f5f85-p827w Readiness probe failed: HTTP probe failed with statuscode: 503 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-844db7-6f7649b585pscw Readiness probe failed: timeout: failed to connect service "10.133.1.70:50052" within 1s: context deadline exceeded 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-844db7-d587bc6mhg Readiness probe failed: timeout: failed to connect service "10.133.1.69:50052" within 1s: context deadline exceeded 8m47s Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Readiness probe failed: Get "http://10.132.1.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m47s Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-qgfcs Readiness probe error: Get "http://10.132.1.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m47s Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Readiness probe failed: Get "http://10.133.1.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m47s Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-t27sn Readiness probe error: Get "http://10.133.1.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m47s Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Readiness probe error: Get "http://10.132.1.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m47s Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-f92ld Readiness probe failed: Get "http://10.132.1.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m15s Normal ScalingReplicaSet deployment/e2e-worker-single-main-8407e9-openshift-default Scaled up replica set e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7c from 0 to 1 8m15s Normal SuccessfulCreate replicaset/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7c Created pod: e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj 8m15s Normal AddedLabel gateway/e2e-worker-single-main-8407e9 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-8407e9 8m14s Normal AddedInterface pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Add eth0 [10.133.1.75/23] from ovn-kubernetes 8m10s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-8407e9-544db547f Created pod: payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw 8m10s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c Error creating: pods "payload-processing-e2e-worker-single-main-8407e9-7b7d8679c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-8407e9: serviceaccount "payload-processing-e2e-worker-single-main-8407e9" not found 8m10s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-8407e9 Scaled up replica set payload-pre-processing-e2e-worker-single-main-8407e9-544db547f from 0 to 1 8m10s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c Created pod: payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x 8m10s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-8407e9-544db547f Error creating: pods "payload-pre-processing-e2e-worker-single-main-8407e9-544db547f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-8407e9: serviceaccount "payload-processing-e2e-worker-single-main-8407e9" not found 8m10s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-8407e9 Scaled up replica set payload-processing-e2e-worker-single-main-8407e9-7b7d8679c from 0 to 1 8m9s Normal Created pod/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x Container created 8m9s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m9s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw Add eth0 [10.133.1.76/23] from ovn-kubernetes 8m9s Normal Started pod/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x Container started 8m9s Normal Started pod/payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw Container started 8m9s Normal Created pod/payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw Container created 8m9s Normal Pulled pod/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m9s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x Add eth0 [10.134.1.16/23] from ovn-kubernetes 7m58s Warning ProbeError pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Readiness probe error: Get "http://10.133.1.75:15021/healthz/ready": dial tcp 10.133.1.75:15021: connect: connection refused... 7m58s Warning Unhealthy pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Readiness probe failed: Get "http://10.133.1.75:15021/healthz/ready": dial tcp 10.133.1.75:15021: connect: connection refused 7m57s Normal Pulled pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 7m57s Normal Created pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Container created 7m57s Normal Started pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Container started 7m52s Warning Unhealthy pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Startup probe failed: Get "http://10.133.1.75:15021/healthz/ready": dial tcp 10.133.1.75:15021: connect: connection refused 7m52s Warning ProbeError pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Startup probe error: Get "http://10.133.1.75:15021/healthz/ready": dial tcp 10.133.1.75:15021: connect: connection refused... 7m15s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-8407e9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m15s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-8407e9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m3s Normal Killing pod/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x Stopping container payload-processing 7m3s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw Stopping container payload-pre-processing 6m55s Normal Killing pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Stopping container istio-proxy 6m51s Warning ProbeError pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Readiness probe error: HTTP probe failed with statuscode: 503... 6m51s Warning Unhealthy pod/e2e-worker-single-main-8407e9-openshift-default-5cf8b85f7ckdtlj Readiness probe failed: HTTP probe failed with statuscode: 503 6m38s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-8407e9-7b7d8679c2rs6x Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m38s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-8407e9-544db86kfw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m47s Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Readiness probe error: Get "http://10.133.1.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m47s Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-6rwmb Readiness probe failed: Get "http://10.133.1.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m47s Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Readiness probe error: Get "http://10.133.1.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m47s Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-68qbd Readiness probe failed: Get "http://10.133.1.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m20s Normal ScalingReplicaSet deployment/e2e-ait-4554a17c-openshift-default Scaled up replica set e2e-ait-4554a17c-openshift-default-757d594ccc from 0 to 1 5m20s Normal AddedLabel gateway/e2e-ait-4554a17c Added label istio.io/rev=openshift-gateway to gateway e2e-ait-4554a17c 5m20s Normal AddedInterface pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Add eth0 [10.133.1.78/23] from ovn-kubernetes 5m20s Normal SuccessfulCreate replicaset/e2e-ait-4554a17c-openshift-default-757d594ccc Created pod: e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs 5m16s Normal Created pod/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-x8n6g Container created 5m16s Normal Created pod/payload-processing-e2e-ait-4554a17c-bfdd8df5-89k9l Container created 5m16s Normal Started pod/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-x8n6g Container started 5m16s Normal Pulled pod/payload-processing-e2e-ait-4554a17c-bfdd8df5-89k9l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m16s Normal Started pod/payload-processing-e2e-ait-4554a17c-bfdd8df5-89k9l Container started 5m16s Warning FailedCreate replicaset/payload-processing-e2e-ait-4554a17c-bfdd8df5 Error creating: pods "payload-processing-e2e-ait-4554a17c-bfdd8df5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4554a17c: serviceaccount "payload-processing-e2e-ait-4554a17c" not found 5m16s Normal AddedInterface pod/payload-processing-e2e-ait-4554a17c-bfdd8df5-89k9l Add eth0 [10.134.1.22/23] from ovn-kubernetes 5m16s Normal AddedInterface pod/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-x8n6g Add eth0 [10.133.1.79/23] from ovn-kubernetes 5m16s Normal Pulled pod/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-x8n6g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m16s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-4554a17c-bfdd8df5 Created pod: payload-processing-e2e-ait-4554a17c-bfdd8df5-89k9l 5m16s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-4554a17c Scaled up replica set payload-processing-e2e-ait-4554a17c-bfdd8df5 from 0 to 1 5m16s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-4554a17c Scaled up replica set payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf from 0 to 1 5m16s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf Created pod: payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-x8n6g 5m16s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf Error creating: pods "payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4554a17c: serviceaccount "payload-processing-e2e-ait-4554a17c" not found 5m3s Normal Pulled pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m3s Warning Unhealthy pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Readiness probe failed: Get "http://10.133.1.78:15021/healthz/ready": dial tcp 10.133.1.78:15021: connect: connection refused 5m3s Warning ProbeError pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Readiness probe error: Get "http://10.133.1.78:15021/healthz/ready": dial tcp 10.133.1.78:15021: connect: connection refused... 5m3s Normal Started pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Container started 5m3s Normal Created pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Container created 4m58s Warning ProbeError pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Startup probe error: Get "http://10.133.1.78:15021/healthz/ready": dial tcp 10.133.1.78:15021: connect: connection refused... 4m58s Warning Unhealthy pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Startup probe failed: Get "http://10.133.1.78:15021/healthz/ready": dial tcp 10.133.1.78:15021: connect: connection refused 4m52s Normal Killing pod/payload-processing-e2e-ait-4554a17c-bfdd8df5-89k9l Stopping container payload-processing 4m52s Normal Killing pod/payload-pre-processing-e2e-ait-4554a17c-5776dfb9cf-x8n6g Stopping container payload-pre-processing 4m50s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-4554a17c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m50s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-4554a17c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m45s Normal Killing pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Stopping container istio-proxy 4m44s Normal AddedLabel gateway/e2e-authcfg-http-33c6d1 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-33c6d1 4m43s Normal ScalingReplicaSet deployment/e2e-authcfg-http-33c6d1-openshift-default Scaled up replica set e2e-authcfg-http-33c6d1-openshift-default-5547cf548f from 0 to 1 4m43s Normal SuccessfulCreate replicaset/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f Created pod: e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww 4m43s Normal AddedInterface pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Add eth0 [10.133.1.82/23] from ovn-kubernetes 4m42s Warning Unhealthy pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Readiness probe failed: HTTP probe failed with statuscode: 503 4m42s Warning ProbeError pod/e2e-ait-4554a17c-openshift-default-757d594ccc-gbdvs Readiness probe error: HTTP probe failed with statuscode: 503... 4m39s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-33c6d1 Scaled up replica set payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887 from 0 to 1 4m38s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-6jzkt Add eth0 [10.133.1.84/23] from ovn-kubernetes 4m38s Normal Pulled pod/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-9hndx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m38s Normal Created pod/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-6jzkt Container created 4m38s Normal Started pod/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-6jzkt Container started 4m38s Normal Created pod/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-9hndx Container created 4m38s Normal Started pod/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-9hndx Container started 4m38s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-6jzkt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m38s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887 Error creating: pods "payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-33c6d1: serviceaccount "payload-processing-e2e-authcfg-http-33c6d1" not found 4m38s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887 Created pod: payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-9hndx 4m38s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-33c6d1 Scaled up replica set payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495 from 0 to 1 4m38s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-9hndx Add eth0 [10.134.1.23/23] from ovn-kubernetes 4m38s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495 Created pod: payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-6jzkt 4m38s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495 Error creating: pods "payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-33c6d1: serviceaccount "payload-processing-e2e-authcfg-http-33c6d1" not found 4m26s Normal Pulled pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m26s Normal Created pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Container created 4m26s Normal Started pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Container started 4m26s Warning ProbeError pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Readiness probe error: Get "http://10.133.1.82:15021/healthz/ready": dial tcp 10.133.1.82:15021: connect: connection refused... 4m26s Warning Unhealthy pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Readiness probe failed: Get "http://10.133.1.82:15021/healthz/ready": dial tcp 10.133.1.82:15021: connect: connection refused 4m20s Warning ProbeError pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Startup probe error: Get "http://10.133.1.82:15021/healthz/ready": dial tcp 10.133.1.82:15021: connect: connection refused... 4m20s Warning Unhealthy pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Startup probe failed: Get "http://10.133.1.82:15021/healthz/ready": dial tcp 10.133.1.82:15021: connect: connection refused 3m58s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-33c6d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m58s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-33c6d1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 2m54s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-33c6d1-6c98645495-6jzkt Stopping container payload-pre-processing 2m54s Normal Killing pod/payload-processing-e2e-authcfg-http-33c6d1-7b58bf8887-9hndx Stopping container payload-processing 2m40s Normal Killing pod/e2e-authcfg-http-33c6d1-openshift-default-5547cf548f-btpww Stopping container istio-proxy 2m35s Warning Unhealthy pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Readiness probe failed: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m35s Warning ProbeError pod/maas-default-gateway-openshift-default-75d7c785f-t5m2j Readiness probe error: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...