=== events in openshift-ingress (2026-10-31T01:43:32Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 46m Normal Scheduled pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Successfully assigned openshift-ingress/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 to ip-10-0-129-199.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-fddcb Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-fddcb to ip-10-0-132-66.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 to ip-10-0-128-253.ec2.internal 21m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s to ip-10-0-129-199.ec2.internal 60m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb to ip-10-0-132-66.ec2.internal 21m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c to ip-10-0-132-66.ec2.internal 21m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg to ip-10-0-132-66.ec2.internal 65m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx to ip-10-0-128-253.ec2.internal 5m29s Normal Scheduled pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Successfully assigned openshift-ingress/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs to ip-10-0-129-199.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm to ip-10-0-128-253.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz to ip-10-0-129-199.ec2.internal 63m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp to ip-10-0-129-199.ec2.internal 65m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq to ip-10-0-128-253.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc to ip-10-0-129-199.ec2.internal 59m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj to ip-10-0-128-253.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b Successfully assigned openshift-ingress/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b to ip-10-0-132-66.ec2.internal 60m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 to ip-10-0-129-199.ec2.internal 40m Normal Scheduled pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Successfully assigned openshift-ingress/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt to ip-10-0-129-199.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 to ip-10-0-132-66.ec2.internal 59m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x to ip-10-0-128-253.ec2.internal 22m Normal Scheduled pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Successfully assigned openshift-ingress/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j to ip-10-0-128-253.ec2.internal 59m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj to ip-10-0-128-253.ec2.internal 84m Normal Scheduled pod/router-default-55f45b7759-45twl Successfully assigned openshift-ingress/router-default-55f45b7759-45twl to ip-10-0-132-66.ec2.internal 85m Warning FailedScheduling pod/router-default-55f45b7759-45twl no nodes available to schedule pods 59m Normal Scheduled pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Successfully assigned openshift-ingress/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs to ip-10-0-132-66.ec2.internal 57m Normal Scheduled pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Successfully assigned openshift-ingress/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw to ip-10-0-132-66.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Successfully assigned openshift-ingress/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd to ip-10-0-128-253.ec2.internal 62m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt to ip-10-0-132-66.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 to ip-10-0-132-66.ec2.internal 60m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x to ip-10-0-132-66.ec2.internal 65m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp to ip-10-0-128-253.ec2.internal 56m Normal Scheduled pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Successfully assigned openshift-ingress/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Successfully assigned openshift-ingress/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 to ip-10-0-132-66.ec2.internal 55m Normal Scheduled pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd Successfully assigned openshift-ingress/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd to ip-10-0-132-66.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l to ip-10-0-129-199.ec2.internal 60m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf to ip-10-0-132-66.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm to ip-10-0-132-66.ec2.internal 39m Normal Scheduled pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Successfully assigned openshift-ingress/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg to ip-10-0-132-66.ec2.internal 65m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 to ip-10-0-128-253.ec2.internal 58m Normal Scheduled pod/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd Successfully assigned openshift-ingress/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd to ip-10-0-128-253.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg Successfully assigned openshift-ingress/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg to ip-10-0-132-66.ec2.internal 60m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb to ip-10-0-132-66.ec2.internal 62m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 to ip-10-0-128-253.ec2.internal 60m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn to ip-10-0-129-199.ec2.internal 59m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 to ip-10-0-132-66.ec2.internal 89m Warning FailedScheduling pod/router-default-55f45b7759-45twl no nodes available to schedule pods 65m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 to ip-10-0-128-253.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg to ip-10-0-132-66.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx to ip-10-0-128-253.ec2.internal 59m Normal Scheduled pod/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq Successfully assigned openshift-ingress/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq to ip-10-0-128-253.ec2.internal 54m Normal Scheduled pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz Successfully assigned openshift-ingress/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz to ip-10-0-132-66.ec2.internal 26m Normal Scheduled pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Successfully assigned openshift-ingress/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr to ip-10-0-129-199.ec2.internal 29m Normal Scheduled pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s Successfully assigned openshift-ingress/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s to ip-10-0-128-253.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx to ip-10-0-128-253.ec2.internal 60m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p to ip-10-0-129-199.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Successfully assigned openshift-ingress/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf to ip-10-0-132-66.ec2.internal 60m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 to ip-10-0-132-66.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Successfully assigned openshift-ingress/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx to ip-10-0-132-66.ec2.internal 55m Normal Scheduled pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Successfully assigned openshift-ingress/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf to ip-10-0-129-199.ec2.internal 62m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 to ip-10-0-128-253.ec2.internal 41m Normal Scheduled pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Successfully assigned openshift-ingress/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Successfully assigned openshift-ingress/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls to ip-10-0-132-66.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj to ip-10-0-129-199.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl to ip-10-0-132-66.ec2.internal 61m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 to ip-10-0-129-199.ec2.internal 32m Normal Scheduled pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Successfully assigned openshift-ingress/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm to ip-10-0-129-199.ec2.internal 60m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Successfully assigned openshift-ingress/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m to ip-10-0-132-66.ec2.internal 29m Normal Scheduled pod/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 Successfully assigned openshift-ingress/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 to ip-10-0-129-199.ec2.internal 59m Normal Scheduled pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Successfully assigned openshift-ingress/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 to ip-10-0-128-253.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Successfully assigned openshift-ingress/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg to ip-10-0-129-199.ec2.internal 29m Normal Scheduled pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Successfully assigned openshift-ingress/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 to ip-10-0-132-66.ec2.internal 81m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-8dkw7 to ip-10-0-128-253.ec2.internal 31m Normal Scheduled pod/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq Successfully assigned openshift-ingress/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq to ip-10-0-132-66.ec2.internal 79m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-frj8v Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-frj8v to ip-10-0-128-253.ec2.internal 58m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-s7c69 to ip-10-0-132-66.ec2.internal 53m Normal Scheduled pod/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv Successfully assigned openshift-ingress/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl to ip-10-0-128-253.ec2.internal 60m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 to ip-10-0-128-253.ec2.internal 34m Normal Scheduled pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Successfully assigned openshift-ingress/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j to ip-10-0-128-253.ec2.internal 60m Normal Scheduled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Successfully assigned openshift-ingress/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt to ip-10-0-132-66.ec2.internal 36m Normal Scheduled pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Successfully assigned openshift-ingress/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s to ip-10-0-132-66.ec2.internal 52m Normal Scheduled pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Successfully assigned openshift-ingress/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms to ip-10-0-128-253.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-sgntd Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-sgntd to ip-10-0-129-199.ec2.internal 62m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 to ip-10-0-129-199.ec2.internal 56m Normal Scheduled pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Successfully assigned openshift-ingress/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r to ip-10-0-132-66.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds Successfully assigned openshift-ingress/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds to ip-10-0-129-199.ec2.internal 30m Normal Scheduled pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Successfully assigned openshift-ingress/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn to ip-10-0-132-66.ec2.internal 56m Normal Scheduled pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Successfully assigned openshift-ingress/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd to ip-10-0-128-253.ec2.internal 52m Normal Scheduled pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Successfully assigned openshift-ingress/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt to ip-10-0-128-253.ec2.internal 59m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b to ip-10-0-132-66.ec2.internal 80m Normal Scheduled pod/kube-auth-proxy-94bcdf98f-g9zcg Successfully assigned openshift-ingress/kube-auth-proxy-94bcdf98f-g9zcg to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm to ip-10-0-129-199.ec2.internal 24m Normal Scheduled pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv Successfully assigned openshift-ingress/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv to ip-10-0-129-199.ec2.internal 80m Normal Scheduled pod/kube-auth-proxy-94bcdf98f-xh2jv Successfully assigned openshift-ingress/kube-auth-proxy-94bcdf98f-xh2jv to ip-10-0-129-199.ec2.internal 24m Normal Scheduled pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Successfully assigned openshift-ingress/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp to ip-10-0-129-199.ec2.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-4qhrd to ip-10-0-128-253.ec2.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-62hfr to ip-10-0-129-199.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-6bxb6 to ip-10-0-129-199.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-8gm8p to ip-10-0-132-66.ec2.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w2-735123-8466d8d68f-ssg4w Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-735123-8466d8d68f-ssg4w to ip-10-0-129-199.ec2.internal 50m Normal Scheduled pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Successfully assigned openshift-ingress/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d to ip-10-0-128-253.ec2.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-8zxnz to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Successfully assigned openshift-ingress/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 to ip-10-0-129-199.ec2.internal 63m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd to ip-10-0-128-253.ec2.internal 81m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-97mst Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-97mst to ip-10-0-132-66.ec2.internal 55m Normal Scheduled pod/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh Successfully assigned openshift-ingress/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh to ip-10-0-128-253.ec2.internal 48m Normal Scheduled pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt Successfully assigned openshift-ingress/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt to ip-10-0-128-253.ec2.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-97z9h to ip-10-0-129-199.ec2.internal 45m Normal Scheduled pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Successfully assigned openshift-ingress/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv to ip-10-0-132-66.ec2.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj to ip-10-0-129-199.ec2.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-c7jth to ip-10-0-132-66.ec2.internal 39m Normal Scheduled pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Successfully assigned openshift-ingress/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w to ip-10-0-132-66.ec2.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-hk9qf to ip-10-0-128-253.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt Successfully assigned openshift-ingress/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt to ip-10-0-129-199.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-hlnsl to ip-10-0-128-253.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-ldf84 to ip-10-0-129-199.ec2.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-mbzdk to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm to ip-10-0-129-199.ec2.internal 12m Normal Scheduled pod/payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 to ip-10-0-128-253.ec2.internal 8m32s Normal Scheduled pod/payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc to ip-10-0-128-253.ec2.internal 54m Normal Scheduled pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Successfully assigned openshift-ingress/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms to ip-10-0-129-199.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x to ip-10-0-129-199.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw to ip-10-0-128-253.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc to ip-10-0-128-253.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf to ip-10-0-129-199.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 to ip-10-0-128-253.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh to ip-10-0-132-66.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr to ip-10-0-128-253.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl to ip-10-0-128-253.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb to ip-10-0-128-253.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj to ip-10-0-128-253.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-qrppq Successfully assigned openshift-ingress/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-qrppq to ip-10-0-128-253.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg Successfully assigned openshift-ingress/payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg to ip-10-0-129-199.ec2.internal 54m Normal Scheduled pod/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 Successfully assigned openshift-ingress/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 to ip-10-0-132-66.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-jmtjc Successfully assigned openshift-ingress/payload-processing-e2e-mt-208e9f32-7f6679cdf7-jmtjc to ip-10-0-128-253.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln to ip-10-0-132-66.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 Successfully assigned openshift-ingress/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Successfully assigned openshift-ingress/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 to ip-10-0-132-66.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 to ip-10-0-128-253.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 Successfully assigned openshift-ingress/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf to ip-10-0-132-66.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-nqx6q to ip-10-0-129-199.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-qq9v6 to ip-10-0-129-199.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-migrate-f598cd05-867485b665-k6zsw Successfully assigned openshift-ingress/payload-processing-e2e-migrate-f598cd05-867485b665-k6zsw to ip-10-0-132-66.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-d68696-884f5657-t256p Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-d68696-884f5657-t256p to ip-10-0-128-253.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-d68696-5d76bd94f8-t4rlr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-d68696-5d76bd94f8-t4rlr to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m Successfully assigned openshift-ingress/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m to ip-10-0-129-199.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-r6vv5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-r6vv5 to ip-10-0-129-199.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-b-8f0029-5b558588db-d5gpn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-8f0029-5b558588db-d5gpn to ip-10-0-129-199.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-9l7xc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-9l7xc to ip-10-0-129-199.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-8d251f-757d49dcf5-8lzp7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-8d251f-757d49dcf5-8lzp7 to ip-10-0-128-253.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-b-75ead8-7c5767987b-gp4gv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-75ead8-7c5767987b-gp4gv to ip-10-0-132-66.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-b-75ead8-5b87c69f95-vt2vm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-75ead8-5b87c69f95-vt2vm to ip-10-0-128-253.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-b-33d44f-b69895595-65qmw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-33d44f-b69895595-65qmw to ip-10-0-128-253.ec2.internal 52m Normal Scheduled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Successfully assigned openshift-ingress/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg to ip-10-0-129-199.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-33d44f-694fc9d78d-wc2mq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-33d44f-694fc9d78d-wc2mq to ip-10-0-129-199.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv to ip-10-0-128-253.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-mt8xz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-mt8xz to ip-10-0-129-199.ec2.internal 31m Normal Scheduled pod/payload-processing-e2e-ipp-a-e41918-7f99fb8c89-7lq8d Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e41918-7f99fb8c89-7lq8d to ip-10-0-128-253.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-e41918-766c7f5686-8mf57 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e41918-766c7f5686-8mf57 to ip-10-0-128-253.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-ipp-a-73f915-78fccdbd7f-98rsg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-73f915-78fccdbd7f-98rsg to ip-10-0-129-199.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx to ip-10-0-132-66.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-4207cd-f974fcd45-22f7w Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-4207cd-f974fcd45-22f7w to ip-10-0-128-253.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-4207cd-55648f6b98-x88sq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-4207cd-55648f6b98-x88sq to ip-10-0-128-253.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-114c95-7c5799bc88-w2b2g Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-114c95-7c5799bc88-w2b2g to ip-10-0-128-253.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-114c95-795d94c84c-gsbds Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-114c95-795d94c84c-gsbds to ip-10-0-129-199.ec2.internal 49m Normal Scheduled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 Successfully assigned openshift-ingress/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd to ip-10-0-128-253.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj to ip-10-0-129-199.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-4z7ng Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-4z7ng to ip-10-0-128-253.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-b-6f8856-647b85df9c-tbspl Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-6f8856-647b85df9c-tbspl to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-b-609843-b8d75c975-frvqn Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-609843-b8d75c975-frvqn to ip-10-0-128-253.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-4f4e69-5f8fb86855-mzzfn Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-4f4e69-5f8fb86855-mzzfn to ip-10-0-132-66.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp Successfully assigned openshift-ingress/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp to ip-10-0-128-253.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc to ip-10-0-128-253.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k to ip-10-0-129-199.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-a-762753-64f6f4597c-vnz4m Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-762753-64f6f4597c-vnz4m to ip-10-0-129-199.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd to ip-10-0-128-253.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 to ip-10-0-128-253.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck Successfully assigned openshift-ingress/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck to ip-10-0-132-66.ec2.internal 60m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh to ip-10-0-128-253.ec2.internal 29m Normal Scheduled pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt Successfully assigned openshift-ingress/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk to ip-10-0-129-199.ec2.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-vd2tg to ip-10-0-129-199.ec2.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-vlrk2 to ip-10-0-132-66.ec2.internal 35m Normal Scheduled pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Successfully assigned openshift-ingress/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 to ip-10-0-128-253.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-zhprr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-zhprr to ip-10-0-132-66.ec2.internal 49m Normal Scheduled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz Successfully assigned openshift-ingress/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz to ip-10-0-129-199.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-derive-c2cd8283-587685db69-76crd Successfully assigned openshift-ingress/payload-processing-e2e-derive-c2cd8283-587685db69-76crd to ip-10-0-128-253.ec2.internal 4m46s Normal Scheduled pod/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-pf5zz Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-pf5zz to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-api-b-bdb61f-764449d9c5-86l2g Successfully assigned openshift-ingress/payload-processing-e2e-api-b-bdb61f-764449d9c5-86l2g to ip-10-0-129-199.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-api-b-82d887-76f55bc4f5-p7m4j Successfully assigned openshift-ingress/payload-processing-e2e-api-b-82d887-76f55bc4f5-p7m4j to ip-10-0-128-253.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-api-b-3d21c7-588788f9c-h9x65 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-3d21c7-588788f9c-h9x65 to ip-10-0-128-253.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-api-b-3b1945-577dddb78f-trzq2 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-3b1945-577dddb78f-trzq2 to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-api-a-e166ca-7974df756c-ltngf Successfully assigned openshift-ingress/payload-processing-e2e-api-a-e166ca-7974df756c-ltngf to ip-10-0-129-199.ec2.internal 49m Normal Scheduled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm Successfully assigned openshift-ingress/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm to ip-10-0-132-66.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-api-a-bbb902-65d446d445-znrq5 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-bbb902-65d446d445-znrq5 to ip-10-0-128-253.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-a-68f412-5c85fb46cc-nhxzf Successfully assigned openshift-ingress/payload-processing-e2e-api-a-68f412-5c85fb46cc-nhxzf to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-api-a-14e039-9bcfbb5f9-2hcl5 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-14e039-9bcfbb5f9-2hcl5 to ip-10-0-128-253.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-zrjvf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-zrjvf to ip-10-0-128-253.ec2.internal 60m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg to ip-10-0-128-253.ec2.internal 44m Normal Scheduled pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Successfully assigned openshift-ingress/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt to ip-10-0-129-199.ec2.internal 50m Normal Scheduled pod/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz Successfully assigned openshift-ingress/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz to ip-10-0-132-66.ec2.internal 81m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-84pz9 Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-55f56bc75f-84pz9 to ip-10-0-132-66.ec2.internal 45m Normal Scheduled pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Successfully assigned openshift-ingress/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t to ip-10-0-129-199.ec2.internal 64m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 to ip-10-0-129-199.ec2.internal 12m Normal Scheduled pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Successfully assigned openshift-ingress/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj to ip-10-0-128-253.ec2.internal 59m Normal Scheduled pod/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-svst6 Successfully assigned openshift-ingress/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-svst6 to ip-10-0-128-253.ec2.internal 40m Normal Scheduled pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Successfully assigned openshift-ingress/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w to ip-10-0-132-66.ec2.internal 60m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l to ip-10-0-129-199.ec2.internal 66m Normal Scheduled pod/payload-pre-processing-7b98bb678c-64nxw Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-64nxw to ip-10-0-128-253.ec2.internal 5m22s Normal Scheduled pod/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-n88l8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-n88l8 to ip-10-0-128-253.ec2.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-ndkww Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-ndkww to ip-10-0-132-66.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 to ip-10-0-129-199.ec2.internal 48m Normal Scheduled pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Successfully assigned openshift-ingress/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz to ip-10-0-128-253.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-ait-b08f569a-5bfc7f99cc-ttlzp Successfully assigned openshift-ingress/payload-processing-e2e-ait-b08f569a-5bfc7f99cc-ttlzp to ip-10-0-129-199.ec2.internal 62m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s to ip-10-0-129-199.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz Successfully assigned openshift-ingress/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz to ip-10-0-128-253.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-vk658 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-vk658 to ip-10-0-129-199.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq Successfully assigned openshift-ingress/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq to ip-10-0-129-199.ec2.internal 36m Normal Scheduled pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 Successfully assigned openshift-ingress/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 to ip-10-0-132-66.ec2.internal 59m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth to ip-10-0-129-199.ec2.internal 8m37s Normal Scheduled pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Successfully assigned openshift-ingress/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 to ip-10-0-128-253.ec2.internal 59m Normal Scheduled pod/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-xk6mm Successfully assigned openshift-ingress/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-xk6mm to ip-10-0-128-253.ec2.internal 5m22s Normal Scheduled pod/payload-processing-e2e-ait-1c206c91-77d45f557c-l6rhp Successfully assigned openshift-ingress/payload-processing-e2e-ait-1c206c91-77d45f557c-l6rhp to ip-10-0-128-253.ec2.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh to ip-10-0-132-66.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c Successfully assigned openshift-ingress/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c to ip-10-0-129-199.ec2.internal 52m Normal Scheduled pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Successfully assigned openshift-ingress/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv to ip-10-0-128-253.ec2.internal 62m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv to ip-10-0-129-199.ec2.internal 54m Normal Scheduled pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Successfully assigned openshift-ingress/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-api-a-14e039-996548779-dvvg4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-14e039-996548779-dvvg4 to ip-10-0-128-253.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-a-68f412-9bf56d555-dcm54 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-68f412-9bf56d555-dcm54 to ip-10-0-128-253.ec2.internal 66m Normal Scheduled pod/payload-processing-55fbbbf9dc-7vfd8 Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-7vfd8 to ip-10-0-129-199.ec2.internal 4m51s Normal Scheduled pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Successfully assigned openshift-ingress/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt to ip-10-0-128-253.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-69lb6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-69lb6 to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-api-a-e166ca-5c477786c9-79vt2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-e166ca-5c477786c9-79vt2 to ip-10-0-129-199.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-hf7c2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-hf7c2 to ip-10-0-128-253.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-d9gb5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-d9gb5 to ip-10-0-128-253.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-b-82d887-567965dcbc-7zpst Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-82d887-567965dcbc-7zpst to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-tw6mt Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-tw6mt to ip-10-0-128-253.ec2.internal 4m46s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-tp78s Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-tp78s to ip-10-0-128-253.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-derive-c2cd8283-546b74d947-5bjpz Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-c2cd8283-546b74d947-5bjpz to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w to ip-10-0-129-199.ec2.internal 62m Normal Scheduled pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Successfully assigned openshift-ingress/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g to ip-10-0-132-66.ec2.internal 80m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-7845f94dd55n5ms Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-7845f94dd55n5ms to ip-10-0-132-66.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Successfully assigned openshift-ingress/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 to ip-10-0-128-253.ec2.internal 60m Normal Scheduled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Successfully assigned openshift-ingress/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 to ip-10-0-132-66.ec2.internal 56m Normal Scheduled pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf Successfully assigned openshift-ingress/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf to ip-10-0-129-199.ec2.internal 57m Normal Scheduled pod/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms Successfully assigned openshift-ingress/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms to ip-10-0-129-199.ec2.internal 62m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh to ip-10-0-129-199.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz to ip-10-0-128-253.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph to ip-10-0-129-199.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs to ip-10-0-129-199.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b to ip-10-0-128-253.ec2.internal 57m Normal Scheduled pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj Successfully assigned openshift-ingress/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj to ip-10-0-128-253.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm to ip-10-0-129-199.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk to ip-10-0-129-199.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 to ip-10-0-128-253.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 to ip-10-0-128-253.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t to ip-10-0-128-253.ec2.internal 48m Normal Scheduled pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 Successfully assigned openshift-ingress/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 to ip-10-0-132-66.ec2.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w to ip-10-0-129-199.ec2.internal 50m Normal Scheduled pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Successfully assigned openshift-ingress/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk to ip-10-0-128-253.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt to ip-10-0-128-253.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 to ip-10-0-128-253.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm to ip-10-0-128-253.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 to ip-10-0-128-253.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-qbn4f Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-qbn4f to ip-10-0-129-199.ec2.internal 52m Normal Scheduled pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Successfully assigned openshift-ingress/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht to ip-10-0-128-253.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-hkk4k Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-hkk4k to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f to ip-10-0-129-199.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-nr6dc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-nr6dc to ip-10-0-128-253.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-ndr4b Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-ndr4b to ip-10-0-128-253.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st to ip-10-0-129-199.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-73f915-d658955ff-lphgt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-73f915-d658955ff-lphgt to ip-10-0-128-253.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-cbh5z Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-cbh5z to ip-10-0-128-253.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-2ln6h Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-2ln6h to ip-10-0-128-253.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-qlmj4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-qlmj4 to ip-10-0-132-66.ec2.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 to ip-10-0-129-199.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 to ip-10-0-128-253.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-gzpqt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-gzpqt to ip-10-0-128-253.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-jnr6j Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-jnr6j to ip-10-0-128-253.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc to ip-10-0-128-253.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-qtr54 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-qtr54 to ip-10-0-128-253.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-9fnjk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-9fnjk to ip-10-0-129-199.ec2.internal 50m Normal Scheduled pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Successfully assigned openshift-ingress/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm to ip-10-0-129-199.ec2.internal 22m Normal Scheduled pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 Successfully assigned openshift-ingress/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 to ip-10-0-128-253.ec2.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl to ip-10-0-129-199.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-x87r2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-x87r2 to ip-10-0-129-199.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-7tcms Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-7tcms to ip-10-0-129-199.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 Successfully assigned openshift-ingress/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 to ip-10-0-128-253.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-ldrrd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-ldrrd to ip-10-0-128-253.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-bwqn2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-bwqn2 to ip-10-0-132-66.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-d2m9p Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-d2m9p to ip-10-0-132-66.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 to ip-10-0-132-66.ec2.internal 43m Normal Scheduled pod/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg Successfully assigned openshift-ingress/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg to ip-10-0-132-66.ec2.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 to ip-10-0-129-199.ec2.internal 12m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k to ip-10-0-128-253.ec2.internal 8m32s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg to ip-10-0-128-253.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx to ip-10-0-128-253.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc to ip-10-0-128-253.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb to ip-10-0-128-253.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 to ip-10-0-129-199.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t to ip-10-0-128-253.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt to ip-10-0-128-253.ec2.internal 48m Normal Scheduled pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Successfully assigned openshift-ingress/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg to ip-10-0-132-66.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb to ip-10-0-128-253.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-nvpzl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-nvpzl to ip-10-0-128-253.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x to ip-10-0-128-253.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b to ip-10-0-128-253.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl to ip-10-0-128-253.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 to ip-10-0-128-253.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-s9bnm Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-s9bnm to ip-10-0-128-253.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-tlrt5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-tlrt5 to ip-10-0-128-253.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 to ip-10-0-129-199.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl Successfully assigned openshift-ingress/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl to ip-10-0-129-199.ec2.internal 65m Normal Scheduled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw Successfully assigned openshift-ingress/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw to ip-10-0-129-199.ec2.internal 89m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-55f45b7759 from 0 to 1 89m Normal SuccessfulCreate replicaset/router-default-55f45b7759 Created pod: router-default-55f45b7759-45twl 88m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 88m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 88m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 84m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 84m Warning FailedMount pod/router-default-55f45b7759-45twl MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 84m Warning FailedMount pod/router-default-55f45b7759-45twl MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 84m Warning FailedMount pod/router-default-55f45b7759-45twl MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 84m Warning FailedMount pod/router-default-55f45b7759-45twl MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 84m Normal Pulled pod/router-default-55f45b7759-45twl Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 84m Normal AddedInterface pod/router-default-55f45b7759-45twl Add eth0 [10.132.0.15/23] from ovn-kubernetes 84m Normal Created pod/router-default-55f45b7759-45twl Container created 84m Normal Started pod/router-default-55f45b7759-45twl Container started 81m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 81m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-8dkw7 81m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 81m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 81m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Add eth0 [10.133.0.17/23] from ovn-kubernetes 81m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.943s (2.943s including waiting). Image size: 185476477 bytes. 81m Normal Created pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Container created 81m Normal Started pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Container started 81m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Readiness probe error: HTTP probe failed with statuscode: 503... 81m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 0 to 1 81m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Readiness probe failed: HTTP probe failed with statuscode: 503 81m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-97mst 81m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-84pz9 Add eth0 [10.132.0.34/23] from ovn-kubernetes 81m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-84pz9 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 81m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-97mst Add eth0 [10.132.0.33/23] from ovn-kubernetes 81m Normal Pulling pod/maas-default-gateway-openshift-default-6886b96f9-97mst Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 81m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 81m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-55f56bc75f from 0 to 1 81m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-55f56bc75f Created pod: openshift-ai-inference-openshift-ai-inference-55f56bc75f-84pz9 81m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 81m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-97mst Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 6.316s (6.316s including waiting). Image size: 183555333 bytes. 81m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-84pz9 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 6.404s (6.404s including waiting). Image size: 183555333 bytes. 81m Normal Started pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-84pz9 Container started 81m Normal Created pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-84pz9 Container created 80m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 80m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 80m Normal SuccessfulCreate replicaset/kube-auth-proxy-94bcdf98f Created pod: kube-auth-proxy-94bcdf98f-g9zcg 80m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-7845f94dd5 from 0 to 1 80m Normal SuccessfulCreate replicaset/kube-auth-proxy-94bcdf98f Created pod: kube-auth-proxy-94bcdf98f-xh2jv 80m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-94bcdf98f from 0 to 2 80m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-7845f94dd5 Created pod: data-science-gateway-data-science-gateway-class-7845f94dd55n5ms 80m Normal Created pod/data-science-gateway-data-science-gateway-class-7845f94dd55n5ms Container created 80m Normal Pulling pod/kube-auth-proxy-94bcdf98f-g9zcg Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 80m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-7845f94dd55n5ms Add eth0 [10.132.0.41/23] from ovn-kubernetes 80m Normal Started pod/data-science-gateway-data-science-gateway-class-7845f94dd55n5ms Container started 80m Normal Pulling pod/kube-auth-proxy-94bcdf98f-xh2jv Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 80m Normal AddedInterface pod/kube-auth-proxy-94bcdf98f-xh2jv Add eth0 [10.134.0.30/23] from ovn-kubernetes 80m Normal AddedInterface pod/kube-auth-proxy-94bcdf98f-g9zcg Add eth0 [10.133.0.19/23] from ovn-kubernetes 80m Normal Pulled pod/data-science-gateway-data-science-gateway-class-7845f94dd55n5ms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 80m Normal Pulled pod/kube-auth-proxy-94bcdf98f-g9zcg Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.449s (2.449s including waiting). Image size: 188904677 bytes. 80m Normal Created pod/kube-auth-proxy-94bcdf98f-g9zcg Container created 80m Normal Started pod/kube-auth-proxy-94bcdf98f-g9zcg Container started 80m Normal Created pod/kube-auth-proxy-94bcdf98f-xh2jv Container created 80m Normal Started pod/kube-auth-proxy-94bcdf98f-xh2jv Container started 80m Normal Pulled pod/kube-auth-proxy-94bcdf98f-xh2jv Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.425s (2.425s including waiting). Image size: 188904677 bytes. 80m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 80m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 80m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 80m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 79m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 79m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-frj8v Container started 79m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-frj8v 79m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-frj8v Container created 79m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-frj8v Add eth0 [10.133.0.20/23] from ovn-kubernetes 79m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-frj8v Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 79m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-frj8v Readiness probe error: HTTP probe failed with statuscode: 503... 79m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-frj8v Readiness probe failed: HTTP probe failed with statuscode: 503 79m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-8dkw7 79m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-8dkw7 Stopping container discovery 79m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 78m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 78m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 78m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 78m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-97mst Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-97mst Container created 68m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-97mst Container started 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-97mst Startup probe failed: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused 68m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-97mst Startup probe error: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused... 67m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 1 to 2 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-vd2tg 66m Normal Pulling pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Add eth0 [10.134.0.40/23] from ovn-kubernetes 66m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Container started 66m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Container created 66m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.207s (3.207s including waiting). Image size: 183555333 bytes. 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 66m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 66m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 66m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 66m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 66m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 66m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-64nxw 66m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-7vfd8 66m Normal Pulling pod/payload-pre-processing-7b98bb678c-64nxw Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-97z9h 66m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-64nxw Add eth0 [10.133.0.36/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-8zxnz 66m Normal AddedInterface pod/payload-processing-55fbbbf9dc-7vfd8 Add eth0 [10.134.0.42/23] from ovn-kubernetes 66m Normal Pulling pod/payload-processing-55fbbbf9dc-7vfd8 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Container created 66m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Container started 66m Normal Pulling pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Add eth0 [10.134.0.43/23] from ovn-kubernetes 66m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Add eth0 [10.133.0.37/23] from ovn-kubernetes 66m Normal Started pod/payload-pre-processing-7b98bb678c-64nxw Container started 66m Normal Pulled pod/payload-pre-processing-7b98bb678c-64nxw Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.077s (2.077s including waiting). Image size: 125353965 bytes. 66m Normal Created pod/payload-pre-processing-7b98bb678c-64nxw Container created 66m Normal Pulled pod/payload-processing-55fbbbf9dc-7vfd8 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.009s (3.009s including waiting). Image size: 125353965 bytes. 66m Normal Started pod/payload-processing-55fbbbf9dc-7vfd8 Container started 66m Normal Created pod/payload-processing-55fbbbf9dc-7vfd8 Container created 66m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.509s (3.509s including waiting). Image size: 183555333 bytes. 66m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Container started 66m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Container created 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused 66m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused... 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused 66m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused... 66m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 4 to 8 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-62hfr 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-c7jth 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-hk9qf 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-4qhrd 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Add eth0 [10.132.0.49/23] from ovn-kubernetes 66m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Container created 66m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Container created 66m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Container created 66m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Container started 66m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Container started 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Add eth0 [10.134.0.44/23] from ovn-kubernetes 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Add eth0 [10.133.0.39/23] from ovn-kubernetes 66m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Container created 66m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Container started 66m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Container started 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Add eth0 [10.133.0.40/23] from ovn-kubernetes 66m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused... 66m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused... 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused 66m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 66m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused... 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-vlrk2 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 8 to 9 66m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: 66m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Container created 66m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Add eth0 [10.132.0.50/23] from ovn-kubernetes 66m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Container started 65m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused 65m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled up replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 0 to 1 65m Normal AddedLabel gateway/e2e-worker-w0-671959 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-671959 65m Normal AddedLabel gateway/e2e-worker-w1-b024b5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-b024b5 65m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 65m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 65m Normal AddedLabel gateway/e2e-worker-w6-be301b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-be301b 65m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq 65m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled up replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 0 to 1 65m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled up replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 0 to 1 65m Normal AddedLabel gateway/e2e-worker-w3-daaf26 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-daaf26 65m Normal AddedLabel gateway/e2e-worker-w5-c9e377 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-c9e377 65m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled up replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 0 to 1 65m Normal AddedLabel gateway/e2e-worker-w4-0d8bae Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-0d8bae 65m Normal AddedLabel gateway/e2e-worker-w2-735123 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-735123 65m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx 65m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 65m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Add eth0 [10.133.0.42/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd 65m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx Add eth0 [10.133.0.43/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw 65m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled up replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 0 to 1 65m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled up replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 0 to 1 65m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Add eth0 [10.134.0.46/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled up replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 0 to 1 65m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Add eth0 [10.133.0.41/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Add eth0 [10.134.0.45/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Add eth0 [10.133.0.44/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw Add eth0 [10.134.0.47/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-b024b5 Scaled up replica set payload-processing-e2e-worker-w1-b024b5-54459f766c from 0 to 1 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-b024b5 Scaled up replica set payload-pre-processing-e2e-worker-w1-b024b5-59c565544c from 0 to 1 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-b024b5-54459f766c Error creating: pods "payload-processing-e2e-worker-w1-b024b5-54459f766c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-b024b5: serviceaccount "payload-processing-e2e-worker-w1-b024b5" not found 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c Created pod: payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c Error creating: pods "payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-b024b5: serviceaccount "payload-processing-e2e-worker-w1-b024b5" not found 65m Normal Pulled pod/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal Created pod/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj Container created 65m Normal AddedInterface pod/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj Add eth0 [10.134.0.49/23] from ovn-kubernetes 65m Normal Started pod/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl Container started 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-b024b5-54459f766c Created pod: payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl Add eth0 [10.134.0.48/23] from ovn-kubernetes 65m Normal Started pod/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj Container started 65m Normal Created pod/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl Container created 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-daaf26 Scaled up replica set payload-processing-e2e-worker-w3-daaf26-86c9494789 from 0 to 1 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-daaf26 Scaled up replica set payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf from 0 to 1 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-daaf26-86c9494789 Error creating: pods "payload-processing-e2e-worker-w3-daaf26-86c9494789-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-daaf26: serviceaccount "payload-processing-e2e-worker-w3-daaf26" not found 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf Error creating: pods "payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-daaf26: serviceaccount "payload-processing-e2e-worker-w3-daaf26" not found 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-daaf26-86c9494789 Created pod: payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f Add eth0 [10.134.0.51/23] from ovn-kubernetes 65m Normal Started pod/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f Container started 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf Created pod: payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f 65m Normal Created pod/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f Container created 65m Normal AddedInterface pod/payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm Add eth0 [10.134.0.52/23] from ovn-kubernetes 65m Normal Started pod/payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm Container started 65m Normal Pulled pod/payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal Created pod/payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm Container created 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-735123 Scaled up replica set payload-processing-e2e-worker-w2-735123-8466d8d68f from 0 to 1 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799 Created pod: payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799 Error creating: pods "payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-735123: serviceaccount "payload-processing-e2e-worker-w2-735123" not found 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-735123-8466d8d68f Error creating: pods "payload-processing-e2e-worker-w2-735123-8466d8d68f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-735123: serviceaccount "payload-processing-e2e-worker-w2-735123" not found 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-735123 Scaled up replica set payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799 from 0 to 1 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-735123-8466d8d68f Created pod: payload-processing-e2e-worker-w2-735123-8466d8d68f-ssg4w 65m Normal Created pod/payload-processing-e2e-worker-w2-735123-8466d8d68f-ssg4w Container created 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-be301b-77c7dbd46f Created pod: payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9 Error creating: pods "payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-be301b: serviceaccount "payload-processing-e2e-worker-w6-be301b" not found 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 Add eth0 [10.134.0.53/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9 Created pod: payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w 65m Normal Started pod/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 Container started 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-be301b-77c7dbd46f Error creating: pods "payload-processing-e2e-worker-w6-be301b-77c7dbd46f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-be301b: serviceaccount "payload-processing-e2e-worker-w6-be301b" not found 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-be301b Scaled up replica set payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9 from 0 to 1 65m Normal Started pod/payload-processing-e2e-worker-w2-735123-8466d8d68f-ssg4w Container started 65m Normal AddedInterface pod/payload-processing-e2e-worker-w2-735123-8466d8d68f-ssg4w Add eth0 [10.134.0.54/23] from ovn-kubernetes 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-be301b Scaled up replica set payload-processing-e2e-worker-w6-be301b-77c7dbd46f from 0 to 1 65m Normal Created pod/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 Container created 65m Normal Pulled pod/payload-processing-e2e-worker-w2-735123-8466d8d68f-ssg4w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979 Created pod: payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph 65m Normal Pulled pod/payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal Created pod/payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 Container created 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-c9e377 Scaled up replica set payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c from 0 to 1 65m Normal Started pod/payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 Container started 65m Normal AddedInterface pod/payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 Add eth0 [10.134.0.56/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c Created pod: payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979 Error creating: pods "payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-c9e377: serviceaccount "payload-processing-e2e-worker-w5-c9e377" not found 65m Normal Started pod/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w Container started 65m Normal Created pod/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w Container created 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w Add eth0 [10.134.0.55/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-c9e377 Scaled up replica set payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979 from 0 to 1 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c Error creating: pods "payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-c9e377: serviceaccount "payload-processing-e2e-worker-w5-c9e377" not found 65m Normal Pulled pod/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl Add eth0 [10.133.0.48/23] from ovn-kubernetes 65m Normal Started pod/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl Container started 65m Normal Created pod/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl Container created 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal Created pod/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph Container created 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph Add eth0 [10.134.0.57/23] from ovn-kubernetes 65m Normal Started pod/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph Container started 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-671959-86968d9675 Created pod: payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-671959 Scaled up replica set payload-processing-e2e-worker-w0-671959-6bb586d9d5 from 0 to 1 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-671959 Scaled up replica set payload-pre-processing-e2e-worker-w0-671959-86968d9675 from 0 to 1 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 Add eth0 [10.134.0.58/23] from ovn-kubernetes 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-671959-86968d9675 Error creating: pods "payload-pre-processing-e2e-worker-w0-671959-86968d9675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-671959: serviceaccount "payload-processing-e2e-worker-w0-671959" not found 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-671959-6bb586d9d5 Created pod: payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-671959-6bb586d9d5 Error creating: pods "payload-processing-e2e-worker-w0-671959-6bb586d9d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-671959: serviceaccount "payload-processing-e2e-worker-w0-671959" not found 65m Normal AddedInterface pod/payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm Add eth0 [10.134.0.59/23] from ovn-kubernetes 65m Normal Created pod/payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 Container created 65m Normal Started pod/payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 Container started 65m Normal Pulled pod/payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal Created pod/payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm Container created 65m Normal Started pod/payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm Container started 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467 Error creating: pods "payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-0d8bae: serviceaccount "payload-processing-e2e-worker-w4-0d8bae" not found 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-0d8bae Scaled up replica set payload-pre-processing-e2e-worker-w4-0d8bae-85c849467 from 0 to 1 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7 Error creating: pods "payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-0d8bae: serviceaccount "payload-processing-e2e-worker-w4-0d8bae" not found 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467 Created pod: payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7 Created pod: payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-0d8bae Scaled up replica set payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7 from 0 to 1 65m Normal Created pod/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 Container created 65m Normal Pulled pod/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 Add eth0 [10.134.0.61/23] from ovn-kubernetes 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal Created pod/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w Container created 65m Normal Started pod/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 Container started 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w Add eth0 [10.134.0.60/23] from ovn-kubernetes 65m Normal Started pod/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w Container started 65m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Readiness probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 65m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Readiness probe failed: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 65m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Container created 65m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Container started 65m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused 65m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx Container created 65m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Container started 65m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx Container started 65m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Container created 65m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused... 65m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Container started 65m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Container created 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused... 65m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Container created 65m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Container started 65m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw Container started 65m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw Container created 65m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 65m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused... 65m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused 65m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Container created 65m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Container started 65m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused 65m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 65m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused... 65m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused... 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled up replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 1 to 2 64m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Container started 64m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Container created 64m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 64m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Add eth0 [10.134.0.66/23] from ovn-kubernetes 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 63m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 63m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled up replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 1 to 2 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp 63m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp Add eth0 [10.134.0.67/23] from ovn-kubernetes 63m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp Container created 63m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp Container started 63m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 63m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd 63m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled up replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 1 to 2 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled up replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 1 to 2 63m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Add eth0 [10.133.0.53/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Container started 63m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Container created 63m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Container started 63m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Container created 63m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Add eth0 [10.133.0.54/23] from ovn-kubernetes 63m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 62m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv Add eth0 [10.134.0.74/23] from ovn-kubernetes 62m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled up replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 1 to 2 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 62m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled up replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 1 to 2 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm 62m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv Container started 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 62m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv Container created 62m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled up replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 2 to 4 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh 62m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv 62m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Add eth0 [10.133.0.55/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Add eth0 [10.132.0.72/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl Add eth0 [10.132.0.73/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl Container created 62m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Container started 62m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Container started 62m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Container created 62m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Container created 62m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl Container started 62m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 62m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled up replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 1 to 2 62m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 62m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 Container created 62m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 Container started 62m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 Add eth0 [10.133.0.56/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt 62m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Container started 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 62m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Container created 62m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled up replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 2 to 3 62m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Add eth0 [10.132.0.76/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 62m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 9 to 10 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Add eth0 [10.133.0.57/23] from ovn-kubernetes 62m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Container started 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-mbzdk 62m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Container created 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-xw55s 62m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 62m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled up replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 2 to 4 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb 62m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled up replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 2 to 4 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx 62m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled up replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 4 to 5 62m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 62m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Container created 62m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Container started 62m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Container started 62m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Container created 62m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 Add eth0 [10.133.0.59/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Container created 62m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Container started 62m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s Container started 62m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Add eth0 [10.132.0.78/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Add eth0 [10.133.0.58/23] from ovn-kubernetes 62m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s Container created 62m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh Add eth0 [10.134.0.83/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled up replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 3 to 5 62m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb Container started 62m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb Container created 62m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb Add eth0 [10.134.0.82/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Add eth0 [10.132.0.77/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s Add eth0 [10.134.0.81/23] from ovn-kubernetes 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh 62m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh Container started 62m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh Container created 62m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 Container started 62m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 Container created 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled up replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 2 to 4 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 5; reason: 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj 62m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled up replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 4 to 5 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 5; reason: 62m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 62m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled up replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 4 to 5 61m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz Container created 61m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 Add eth0 [10.133.0.60/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz Container started 61m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Add eth0 [10.132.0.80/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Container started 61m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz Add eth0 [10.133.0.61/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Add eth0 [10.134.0.84/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Container started 61m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 Container created 61m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 Container started 61m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Container created 61m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Container created 61m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 61m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg Container started 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 61m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg Container created 61m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg Add eth0 [10.132.0.82/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 61m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Add eth0 [10.133.0.63/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled up replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 2 to 4 61m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled up replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 5 to 8 61m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd 61m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c Container created 61m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c Add eth0 [10.132.0.83/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Container started 61m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Container created 61m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c Container started 61m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Add eth0 [10.133.0.64/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Container started 61m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Container created 61m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Add eth0 [10.134.0.86/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Container created 61m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Container started 61m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled up replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 4 to 5 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 61m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Add eth0 [10.134.0.88/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled up replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 5 to 6 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 61m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Container created 61m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled up replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 2 to 4 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m 61m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Container started 61m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Container created 61m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf Container started 61m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf Container created 61m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf Add eth0 [10.132.0.84/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Add eth0 [10.132.0.85/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Container started 61m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Container started 61m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Container created 61m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Add eth0 [10.134.0.89/23] from ovn-kubernetes 61m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 61m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled up replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 4 to 7 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled up replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 6 to 10 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 (combined from similar events): Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-55w4l 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-cddfp 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Created pod: e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 61m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Add eth0 [10.133.0.67/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Container created 61m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Container started 61m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l Add eth0 [10.134.0.92/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Add eth0 [10.132.0.86/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Container created 61m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Container started 61m Normal AddedInterface pod/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk Add eth0 [10.134.0.91/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Add eth0 [10.134.0.93/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Add eth0 [10.133.0.68/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Container created 61m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Container started 61m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb Add eth0 [10.132.0.87/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb Container created 61m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb Container started 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled up replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 5 to 10 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 (combined from similar events): Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Created pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm 61m Normal Pulled pod/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk Container created 61m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 Container started 61m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx Container started 61m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm Add eth0 [10.134.0.94/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 Container created 61m Normal Created pod/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l Container created 61m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l Container started 61m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 Add eth0 [10.132.0.88/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm Container created 61m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Container started 61m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm Container started 61m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Container created 61m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch Add eth0 [10.133.0.69/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch Container started 61m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx Add eth0 [10.133.0.70/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx Container created 61m Normal Started pod/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk Container started 61m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch Container created 61m Normal Started pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw Container started 61m Normal Created pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw Container created 61m Normal Pulled pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw Add eth0 [10.132.0.89/23] from ovn-kubernetes 61m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb 61m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled up replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 8 to 10 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 (combined from similar events): Created pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled up replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 5 to 7 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 60m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Container started 60m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l Container started 60m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Container created 60m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l Container created 60m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Container started 60m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Add eth0 [10.133.0.71/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l Add eth0 [10.134.0.96/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Add eth0 [10.132.0.90/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb Add eth0 [10.132.0.91/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Container created 60m Normal Created pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb Container created 60m Normal Started pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb Container started 60m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 60m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh 60m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr 60m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled up replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 5 to 10 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 (combined from similar events): Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf 60m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l 60m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg 60m Normal SuccessfulCreate replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Created pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn 60m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled up replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 4 to 7 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr 60m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 60m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Container created 60m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Add eth0 [10.133.0.74/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Add eth0 [10.133.0.73/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 Add eth0 [10.132.0.95/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 Container created 60m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 Container started 60m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr Add eth0 [10.133.0.75/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Container started 60m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Add eth0 [10.132.0.94/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Container started 60m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr Add eth0 [10.134.0.102/23] from ovn-kubernetes 60m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Container created 60m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Container started 60m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Add eth0 [10.134.0.101/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Container created 60m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Container created 60m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Container started 60m Normal Pulled pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Add eth0 [10.134.0.100/23] from ovn-kubernetes 60m Normal Created pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Container created 60m Normal Started pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Container started 60m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr Container started 60m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr Container created 60m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr Container created 60m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr Container started 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-sgntd 60m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 60m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-sgntd Add eth0 [10.134.0.105/23] from ovn-kubernetes 60m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-sgntd Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 60m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-sgntd Container started 60m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-sgntd Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.725s (4.725s including waiting). Image size: 185476477 bytes. 60m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-sgntd Container created 60m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 (combined from similar events): Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt 60m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled up replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 7 to 10 60m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 60m Normal SuccessfulCreate replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Created pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 60m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt Container created 60m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt Add eth0 [10.133.0.76/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Container started 60m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Add eth0 [10.132.0.99/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt Container started 60m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Container created 60m Normal Created pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Container created 60m Normal Started pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Container started 60m Normal Pulled pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Add eth0 [10.134.0.106/23] from ovn-kubernetes 60m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Readiness probe error: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Readiness probe failed: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Readiness probe failed: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Readiness probe failed: Get "http://10.134.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Readiness probe error: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Readiness probe error: Get "http://10.134.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f (combined from similar events): Created pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth 59m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled up replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 7 to 10 59m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Container created 59m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b Add eth0 [10.132.0.107/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b Container started 59m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Container started 59m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Add eth0 [10.133.0.77/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth Add eth0 [10.134.0.110/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth Container created 59m Normal Started pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth Container started 59m Normal Created pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b Container created 59m Normal Pulled pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 59m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth Startup probe failed: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth Startup probe error: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled up replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 7 to 10 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f (combined from similar events): Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Created pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x 59m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Add eth0 [10.133.0.79/23] from ovn-kubernetes 59m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Container created 59m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Container started 59m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Add eth0 [10.133.0.78/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Container started 59m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Container created 59m Normal AddedInterface pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Add eth0 [10.132.0.112/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Container started 59m Normal Created pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Container created 59m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Readiness probe error: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Readiness probe failed: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal AddedLabel gateway/e2e-ait-d66e61c1 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-d66e61c1 59m Normal SuccessfulCreate replicaset/e2e-ait-d66e61c1-openshift-default-77c7cff769 Created pod: e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq 59m Normal ScalingReplicaSet deployment/e2e-ait-d66e61c1-openshift-default Scaled up replica set e2e-ait-d66e61c1-openshift-default-77c7cff769 from 0 to 1 59m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Readiness probe failed: Get "http://10.133.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Readiness probe error: Get "http://10.133.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Readiness probe error: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Readiness probe failed: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal AddedInterface pod/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq Add eth0 [10.133.0.80/23] from ovn-kubernetes 59m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Readiness probe error: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Readiness probe failed: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Readiness probe error: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Readiness probe failed: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Readiness probe failed: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Readiness probe error: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Readiness probe error: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Readiness probe failed: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Readiness probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Readiness probe error: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Readiness probe failed: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Readiness probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Startup probe failed: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 59m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f Created pod: payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f Error creating: pods "payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-d66e61c1: serviceaccount "payload-processing-e2e-ait-d66e61c1" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-d66e61c1 Scaled up replica set payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f from 0 to 1 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-d66e61c1 Scaled up replica set payload-processing-e2e-ait-d66e61c1-6bc4dd4f59 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59 Created pod: payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-svst6 59m Warning FailedCreate replicaset/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59 Error creating: pods "payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-d66e61c1: serviceaccount "payload-processing-e2e-ait-d66e61c1" not found 59m Normal Pulled pod/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-svst6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Pulling pod/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 59m Normal AddedInterface pod/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-svst6 Add eth0 [10.133.0.81/23] from ovn-kubernetes 59m Normal AddedInterface pod/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh Add eth0 [10.132.0.116/23] from ovn-kubernetes 59m Normal Started pod/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-svst6 Container started 59m Normal Created pod/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-svst6 Container created 59m Normal AddedLabel gateway/e2e-ait-a9461dc0 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-a9461dc0 59m Normal Created pod/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh Container created 59m Normal Pulled pod/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.409s (2.409s including waiting). Image size: 125353965 bytes. 59m Normal Started pod/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh Container started 59m Normal AddedInterface pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Add eth0 [10.132.0.117/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-ait-a9461dc0-openshift-default Scaled up replica set e2e-ait-a9461dc0-openshift-default-7d9bf49df from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-ait-a9461dc0-openshift-default-7d9bf49df Created pod: e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs 59m Normal AddedLabel gateway/e2e-migrate-f598cd05-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-f598cd05-gw 59m Normal ScalingReplicaSet deployment/e2e-migrate-f598cd05-gw-openshift-default Scaled up replica set e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665 Created pod: e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 59m Normal AddedInterface pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Add eth0 [10.133.0.82/23] from ovn-kubernetes 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9 Error creating: pods "payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a9461dc0: serviceaccount "payload-processing-e2e-ait-a9461dc0" not found 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9 Created pod: payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-ndkww 59m Warning FailedCreate replicaset/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7 Error creating: pods "payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a9461dc0: serviceaccount "payload-processing-e2e-ait-a9461dc0" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7 Created pod: payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-xk6mm 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-a9461dc0 Scaled up replica set payload-processing-e2e-ait-a9461dc0-68d7fb8fb7 from 0 to 1 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-a9461dc0 Scaled up replica set payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9 from 0 to 1 59m Normal AddedInterface pod/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-xk6mm Add eth0 [10.133.0.83/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-xk6mm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Started pod/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-xk6mm Container started 59m Normal AddedInterface pod/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-ndkww Add eth0 [10.132.0.119/23] from ovn-kubernetes 59m Normal Started pod/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-ndkww Container started 59m Normal Created pod/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-xk6mm Container created 59m Normal Created pod/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-ndkww Container created 59m Normal Pulled pod/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-ndkww Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-f598cd05 Scaled up replica set payload-processing-e2e-migrate-f598cd05-867485b665 from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-f598cd05 Scaled up replica set payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9 from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-migrate-f598cd05-867485b665 Error creating: pods "payload-processing-e2e-migrate-f598cd05-867485b665-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-f598cd05: serviceaccount "payload-processing-e2e-migrate-f598cd05" not found 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9 Error creating: pods "payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-f598cd05: serviceaccount "payload-processing-e2e-migrate-f598cd05" not found 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-f598cd05-867485b665 Created pod: payload-processing-e2e-migrate-f598cd05-867485b665-k6zsw 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9 Created pod: payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-d2m9p 58m Normal Started pod/payload-processing-e2e-migrate-f598cd05-867485b665-k6zsw Container started 58m Normal Created pod/payload-processing-e2e-migrate-f598cd05-867485b665-k6zsw Container created 58m Normal Started pod/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-d2m9p Container started 58m Normal Pulled pod/payload-processing-e2e-migrate-f598cd05-867485b665-k6zsw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/payload-processing-e2e-migrate-f598cd05-867485b665-k6zsw Add eth0 [10.132.0.122/23] from ovn-kubernetes 58m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-d2m9p Add eth0 [10.132.0.121/23] from ovn-kubernetes 58m Normal Pulled pod/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-d2m9p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-d2m9p Container created 58m Normal SuccessfulCreate replicaset/e2e-ait-b08f569a-openshift-default-5fd5c4857f Created pod: e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a9461dc0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a9461dc0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal ScalingReplicaSet deployment/e2e-ait-b08f569a-openshift-default Scaled up replica set e2e-ait-b08f569a-openshift-default-5fd5c4857f from 0 to 1 58m Normal AddedLabel gateway/e2e-ait-b08f569a Added label istio.io/rev=openshift-gateway to gateway e2e-ait-b08f569a 58m Normal Pulled pod/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd Container created 58m Normal AddedInterface pod/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd Add eth0 [10.133.0.84/23] from ovn-kubernetes 58m Normal Started pod/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd Container started 58m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Readiness probe error: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Readiness probe failed: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Readiness probe error: Get "http://10.134.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Readiness probe failed: Get "http://10.134.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-f598cd05-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-f598cd05-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Readiness probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning ProbeError pod/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Readiness probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Readiness probe error: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Readiness probe error: Get "http://10.132.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Readiness probe failed: Get "http://10.132.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Readiness probe error: Get "http://10.134.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Readiness probe failed: Get "http://10.134.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Readiness probe failed: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj Readiness probe failed: timeout: health rpc did not complete within 1s 58m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl Readiness probe failed: timeout: health rpc did not complete within 1s 58m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Readiness probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Readiness probe failed: Get "http://10.134.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Readiness probe error: Get "http://10.134.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Readiness probe failed: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Readiness probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Readiness probe error: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Readiness probe failed: Get "http://10.134.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Readiness probe error: Get "http://10.134.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-b08f569a-5bfc7f99cc Created pod: payload-processing-e2e-ait-b08f569a-5bfc7f99cc-ttlzp 58m Warning FailedCreate replicaset/payload-processing-e2e-ait-b08f569a-5bfc7f99cc Error creating: pods "payload-processing-e2e-ait-b08f569a-5bfc7f99cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b08f569a: serviceaccount "payload-processing-e2e-ait-b08f569a" not found 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95 Created pod: payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-vk658 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-b08f569a Scaled up replica set payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95 from 0 to 1 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95 Error creating: pods "payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b08f569a: serviceaccount "payload-processing-e2e-ait-b08f569a" not found 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-b08f569a Scaled up replica set payload-processing-e2e-ait-b08f569a-5bfc7f99cc from 0 to 1 58m Normal Created pod/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-vk658 Container created 58m Normal AddedInterface pod/payload-processing-e2e-ait-b08f569a-5bfc7f99cc-ttlzp Add eth0 [10.134.0.116/23] from ovn-kubernetes 58m Normal Pulled pod/payload-processing-e2e-ait-b08f569a-5bfc7f99cc-ttlzp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Pulled pod/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-vk658 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Started pod/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-vk658 Container started 58m Normal AddedInterface pod/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-vk658 Add eth0 [10.134.0.115/23] from ovn-kubernetes 58m Normal Created pod/payload-processing-e2e-ait-b08f569a-5bfc7f99cc-ttlzp Container created 58m Normal Started pod/payload-processing-e2e-ait-b08f569a-5bfc7f99cc-ttlzp Container started 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-d66e61c1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-d66e61c1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-s7c69 58m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 58m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 58m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Add eth0 [10.132.0.125/23] from ovn-kubernetes 58m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 58m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.61s (2.61s including waiting). Image size: 185476477 bytes. 58m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Container created 58m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Container started 58m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Readiness probe error: HTTP probe failed with statuscode: 503... 58m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Readiness probe failed: HTTP probe failed with statuscode: 503 58m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Readiness probe error: Get "http://10.132.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Readiness probe failed: Get "http://10.132.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Readiness probe error: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Readiness probe failed: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Readiness probe failed: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Readiness probe error: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Readiness probe error: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-mbzdk Readiness probe failed: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Normal Killing pod/payload-processing-e2e-ait-d66e61c1-6bc4dd4f59-svst6 Stopping container payload-processing 58m Normal Killing pod/payload-pre-processing-e2e-ait-d66e61c1-7bd4c4589f-8gzxh Stopping container payload-pre-processing 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-b08f569a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Readiness probe error: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-b08f569a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Readiness probe failed: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Readiness probe error: Get "http://10.132.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Readiness probe failed: Get "http://10.132.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Normal Started pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Container started 58m Normal Created pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Container created 58m Normal Pulled pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning Unhealthy pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Readiness probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Readiness probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning Unhealthy pod/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq Readiness probe failed: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq Readiness probe error: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Normal Killing pod/payload-pre-processing-e2e-ait-a9461dc0-66cdd56c9-ndkww Stopping container payload-pre-processing 57m Normal Killing pod/payload-processing-e2e-ait-a9461dc0-68d7fb8fb7-xk6mm Stopping container payload-processing 57m Normal Pulled pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning Unhealthy pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Startup probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 57m Warning ProbeError pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Startup probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 57m Normal Created pod/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq Container created 57m Normal Started pod/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq Container started 57m Normal Started pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Container started 57m Normal Created pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Container created 57m Normal AddedLabel gateway/e2e-derive-c2cd8283 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-c2cd8283 57m Normal Killing pod/e2e-ait-d66e61c1-openshift-default-77c7cff769-c69sq Stopping container istio-proxy 57m Normal ScalingReplicaSet deployment/e2e-derive-c2cd8283-openshift-default Scaled up replica set e2e-derive-c2cd8283-openshift-default-59f64f5949 from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-derive-c2cd8283-openshift-default-59f64f5949 Created pod: e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms 57m Normal AddedInterface pod/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms Add eth0 [10.134.0.126/23] from ovn-kubernetes 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a9461dc0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a9461dc0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning Unhealthy pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 57m Warning ProbeError pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 57m Warning FailedCreate replicaset/payload-processing-e2e-derive-c2cd8283-587685db69 Error creating: pods "payload-processing-e2e-derive-c2cd8283-587685db69-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c2cd8283: serviceaccount "payload-processing-e2e-derive-c2cd8283" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-c2cd8283 Scaled up replica set payload-pre-processing-e2e-derive-c2cd8283-546b74d947 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-c2cd8283-546b74d947 Created pod: payload-pre-processing-e2e-derive-c2cd8283-546b74d947-5bjpz 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-c2cd8283-546b74d947 Error creating: pods "payload-pre-processing-e2e-derive-c2cd8283-546b74d947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c2cd8283: serviceaccount "payload-processing-e2e-derive-c2cd8283" not found 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-f598cd05-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-f598cd05-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-c2cd8283-587685db69 Created pod: payload-processing-e2e-derive-c2cd8283-587685db69-76crd 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-c2cd8283 Scaled up replica set payload-processing-e2e-derive-c2cd8283-587685db69 from 0 to 1 57m Normal AddedInterface pod/payload-pre-processing-e2e-derive-c2cd8283-546b74d947-5bjpz Add eth0 [10.133.0.88/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-derive-c2cd8283-546b74d947-5bjpz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Started pod/payload-pre-processing-e2e-derive-c2cd8283-546b74d947-5bjpz Container started 57m Normal AddedInterface pod/payload-processing-e2e-derive-c2cd8283-587685db69-76crd Add eth0 [10.133.0.89/23] from ovn-kubernetes 57m Normal Created pod/payload-pre-processing-e2e-derive-c2cd8283-546b74d947-5bjpz Container created 57m Normal Pulled pod/payload-processing-e2e-derive-c2cd8283-587685db69-76crd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Started pod/payload-processing-e2e-derive-c2cd8283-587685db69-76crd Container started 57m Normal Created pod/payload-processing-e2e-derive-c2cd8283-587685db69-76crd Container created 57m Normal Killing pod/payload-pre-processing-e2e-migrate-f598cd05-6b5cf775f9-d2m9p Stopping container payload-pre-processing 57m Normal Killing pod/payload-processing-e2e-migrate-f598cd05-867485b665-k6zsw Stopping container payload-processing 57m Normal Killing pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Stopping container istio-proxy 57m Warning Unhealthy pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Readiness probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 57m Warning ProbeError pod/e2e-ait-a9461dc0-openshift-default-7d9bf49df-5xwzs Readiness probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 57m Normal Killing pod/e2e-migrate-f598cd05-gw-openshift-default-69c7fc7665-6sd78 Stopping container istio-proxy 57m Normal AddedLabel gateway/e2e-shared-a-w2-36af31 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-36af31 57m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-36af31-openshift-default Scaled up replica set e2e-shared-a-w2-36af31-openshift-default-74f58f595f from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-36af31-openshift-default-74f58f595f Created pod: e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp 57m Normal AddedInterface pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Add eth0 [10.133.0.90/23] from ovn-kubernetes 57m Normal Killing pod/payload-processing-e2e-ait-b08f569a-5bfc7f99cc-ttlzp Stopping container payload-processing 57m Normal Killing pod/payload-pre-processing-e2e-ait-b08f569a-67cfcfcb95-vk658 Stopping container payload-pre-processing 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-36af31 Scaled up replica set payload-processing-e2e-shared-a-w2-36af31-868779fbd5 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f Created pod: payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f Error creating: pods "payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-36af31: serviceaccount "payload-processing-e2e-shared-a-w2-36af31" not found 57m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-36af31-868779fbd5 Error creating: pods "payload-processing-e2e-shared-a-w2-36af31-868779fbd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-36af31: serviceaccount "payload-processing-e2e-shared-a-w2-36af31" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-36af31 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-36af31-868779fbd5 Created pod: payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb 57m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b Add eth0 [10.133.0.91/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb Add eth0 [10.133.0.92/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb Container created 57m Normal Started pod/payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb Container started 57m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b Container created 57m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b Container started 57m Normal Killing pod/e2e-ait-b08f569a-openshift-default-5fd5c4857f-4nlvd Stopping container istio-proxy 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-c2cd8283-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-c2cd8283-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-498580-openshift-default-b66db7999 Created pod: e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj 57m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-498580-openshift-default Scaled up replica set e2e-shared-a-w0-498580-openshift-default-b66db7999 from 0 to 1 57m Normal AddedLabel gateway/e2e-shared-a-w0-498580 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-498580 57m Normal AddedInterface pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj Add eth0 [10.133.0.93/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-498580 Scaled up replica set payload-processing-e2e-shared-a-w0-498580-86c7764565 from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f Error creating: pods "payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-498580: serviceaccount "payload-processing-e2e-shared-a-w0-498580" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f Created pod: payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-498580 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-498580-86c7764565 Error creating: pods "payload-processing-e2e-shared-a-w0-498580-86c7764565-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-498580: serviceaccount "payload-processing-e2e-shared-a-w0-498580" not found 57m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 Add eth0 [10.133.0.95/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-498580-86c7764565 Created pod: payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c 57m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 Container started 57m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 Container created 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-36af31-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal Started pod/payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c Container started 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-36af31-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal Created pod/payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c Container created 57m Normal Pulled pod/payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c Add eth0 [10.133.0.96/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms Container started 56m Normal Created pod/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms Container created 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-c2cd8283-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-c2cd8283-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Killing pod/payload-processing-e2e-derive-c2cd8283-587685db69-76crd Stopping container payload-processing 56m Normal Killing pod/payload-pre-processing-e2e-derive-c2cd8283-546b74d947-5bjpz Stopping container payload-pre-processing 56m Normal SuccessfulCreate replicaset/e2e-mt-208e9f32-openshift-default-77466675fc Created pod: e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 56m Normal AddedLabel gateway/e2e-mt-208e9f32 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-208e9f32 56m Normal ScalingReplicaSet deployment/e2e-mt-208e9f32-openshift-default Scaled up replica set e2e-mt-208e9f32-openshift-default-77466675fc from 0 to 1 56m Warning ProbeError pod/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms Startup probe error: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms Startup probe failed: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused 56m Normal AddedInterface pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Add eth0 [10.134.0.129/23] from ovn-kubernetes 56m Normal AddedLabel gateway/e2e-shared-b-w2-f2bdf3 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-f2bdf3 56m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-f2bdf3-openshift-default Scaled up replica set e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 Created pod: e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv 56m Normal Killing pod/e2e-derive-c2cd8283-openshift-default-59f64f5949-ftzms Stopping container istio-proxy 56m Normal AddedInterface pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Add eth0 [10.133.0.97/23] from ovn-kubernetes 56m Warning FailedCreate replicaset/payload-processing-e2e-mt-208e9f32-7f6679cdf7 Error creating: pods "payload-processing-e2e-mt-208e9f32-7f6679cdf7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-208e9f32: serviceaccount "payload-processing-e2e-mt-208e9f32" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf Created pod: payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-208e9f32 Scaled up replica set payload-processing-e2e-mt-208e9f32-7f6679cdf7 from 0 to 1 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf Error creating: pods "payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-208e9f32: serviceaccount "payload-processing-e2e-mt-208e9f32" not found 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-208e9f32 Scaled up replica set payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf from 0 to 1 56m Normal AddedInterface pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg Add eth0 [10.134.0.131/23] from ovn-kubernetes 56m Normal Created pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg Container created 56m Normal AddedInterface pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 Add eth0 [10.134.0.130/23] from ovn-kubernetes 56m Normal Started pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 Container started 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-208e9f32-7f6679cdf7 Created pod: payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg 56m Normal Pulled pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg Container started 56m Normal Pulled pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 Container created 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-f2bdf3 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f from 0 to 1 56m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689 Error creating: pods "payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-f2bdf3: serviceaccount "payload-processing-e2e-shared-b-w2-f2bdf3" not found 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-f2bdf3 Scaled up replica set payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689 from 0 to 1 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f Error creating: pods "payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-f2bdf3: serviceaccount "payload-processing-e2e-shared-b-w2-f2bdf3" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f Created pod: payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 56m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 Add eth0 [10.134.0.132/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689 Created pod: payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf 56m Normal Pulled pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf Container created 56m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-36af31-openshift-default-74f58f595f Created pod: e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf 56m Normal Started pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf Container started 56m Normal Pulled pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Container created 56m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 Container created 56m Normal AddedInterface pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf Add eth0 [10.134.0.134/23] from ovn-kubernetes 56m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 Container started 56m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf Add eth0 [10.134.0.133/23] from ovn-kubernetes 56m Normal Pulled pod/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Container started 56m Normal Started pod/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf Container started 56m Normal Created pod/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf Container created 56m Warning ProbeError pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf Startup probe failed: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused 56m Warning ProbeError pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf Startup probe error: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused... 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-f2bdf3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-f2bdf3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal AddedLabel gateway/e2e-shared-b-w0-3c90d8 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-3c90d8 56m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-3c90d8-openshift-default Scaled up replica set e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99 from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99 Created pod: e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r 56m Normal AddedInterface pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Add eth0 [10.132.0.133/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c Created pod: payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-3c90d8 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-3c90d8 Scaled up replica set payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c from 0 to 1 56m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c Error creating: pods "payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-3c90d8: serviceaccount "payload-processing-e2e-shared-b-w0-3c90d8" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff Error creating: pods "payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-3c90d8: serviceaccount "payload-processing-e2e-shared-b-w0-3c90d8" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff Created pod: payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt 56m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt Add eth0 [10.133.0.99/23] from ovn-kubernetes 56m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt Container started 56m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt Container created 56m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk Container started 56m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk Add eth0 [10.133.0.98/23] from ovn-kubernetes 56m Normal Pulled pod/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk Container created 56m Normal Pulled pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj Container created 56m Normal Started pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj Container started 56m Warning ProbeError pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj Startup probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj Startup probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-36af31-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-36af31-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal AddedLabel gateway/e2e-shared-a-w6-bcdb4e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-bcdb4e 55m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687 Created pod: e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf 55m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-bcdb4e-openshift-default Scaled up replica set e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687 from 0 to 1 55m Normal AddedInterface pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Add eth0 [10.134.0.137/23] from ovn-kubernetes 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf Error creating: pods "payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-bcdb4e: serviceaccount "payload-processing-e2e-shared-a-w6-bcdb4e" not found 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-bcdb4e Scaled up replica set payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565 from 0 to 1 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-bcdb4e Scaled up replica set payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf Created pod: payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb 55m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565 Error creating: pods "payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-bcdb4e: serviceaccount "payload-processing-e2e-shared-a-w6-bcdb4e" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565 Created pod: payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh 55m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb Container created 55m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Add eth0 [10.134.0.138/23] from ovn-kubernetes 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-f2bdf3-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-f2bdf3-openshift-default Scaled up replica set e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 from 1 to 3 55m Normal Pulled pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb Add eth0 [10.133.0.101/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd Add eth0 [10.132.0.136/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh Add eth0 [10.132.0.135/23] from ovn-kubernetes 55m Normal Started pod/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh Container started 55m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb Container started 55m Normal Pulled pod/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh Container created 55m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 Created pod: e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd 55m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 Created pod: e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf 55m Normal Started pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Container started 55m Warning ProbeError pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Readiness probe error: Get "http://10.134.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Readiness probe failed: Get "http://10.134.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Normal Started pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd Container started 55m Normal Created pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Container created 55m Normal Pulled pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd Container created 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-498580-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-498580-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal Started pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Container started 55m Normal Pulled pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Container created 55m Warning ProbeError pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd Startup probe error: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd Startup probe failed: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused 55m Warning ProbeError pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 55m Normal Started pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Container started 55m Normal Created pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Container created 55m Normal Pulled pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-208e9f32-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal AddedLabel gateway/e2e-api-a-14e039 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-14e039 55m Normal ScalingReplicaSet deployment/e2e-api-a-14e039-openshift-default Scaled up replica set e2e-api-a-14e039-openshift-default-5f7fc9955c from 0 to 1 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-208e9f32-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal SuccessfulCreate replicaset/e2e-api-a-14e039-openshift-default-5f7fc9955c Created pod: e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls 55m Normal AddedInterface pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Add eth0 [10.132.0.142/23] from ovn-kubernetes 55m Warning Unhealthy pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Startup probe failed: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused 55m Warning ProbeError pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Startup probe error: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused... 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-14e039 Scaled up replica set payload-processing-e2e-api-a-14e039-9bcfbb5f9 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-14e039-9bcfbb5f9 Created pod: payload-processing-e2e-api-a-14e039-9bcfbb5f9-2hcl5 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-bcdb4e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedCreate replicaset/payload-processing-e2e-api-a-14e039-9bcfbb5f9 Error creating: pods "payload-processing-e2e-api-a-14e039-9bcfbb5f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-14e039: serviceaccount "payload-processing-e2e-api-a-14e039" not found 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-14e039 Scaled up replica set payload-pre-processing-e2e-api-a-14e039-996548779 from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-14e039-996548779 Error creating: pods "payload-pre-processing-e2e-api-a-14e039-996548779-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-14e039: serviceaccount "payload-processing-e2e-api-a-14e039" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-14e039-996548779 Created pod: payload-pre-processing-e2e-api-a-14e039-996548779-dvvg4 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-bcdb4e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal Created pod/payload-pre-processing-e2e-api-a-14e039-996548779-dvvg4 Container created 55m Normal Pulled pod/payload-pre-processing-e2e-api-a-14e039-996548779-dvvg4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-api-a-14e039-9bcfbb5f9-2hcl5 Container created 55m Normal Started pod/payload-pre-processing-e2e-api-a-14e039-996548779-dvvg4 Container started 55m Normal AddedInterface pod/payload-processing-e2e-api-a-14e039-9bcfbb5f9-2hcl5 Add eth0 [10.133.0.103/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-api-a-14e039-9bcfbb5f9-2hcl5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-processing-e2e-api-a-14e039-9bcfbb5f9-2hcl5 Container started 55m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-14e039-996548779-dvvg4 Add eth0 [10.133.0.102/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/e2e-api-a-e166ca-openshift-default-6584656577 Created pod: e2e-api-a-e166ca-openshift-default-6584656577-7cxfh 55m Normal ScalingReplicaSet deployment/e2e-api-a-e166ca-openshift-default Scaled up replica set e2e-api-a-e166ca-openshift-default-6584656577 from 0 to 1 55m Normal AddedLabel gateway/e2e-api-a-e166ca Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-e166ca 55m Normal AddedInterface pod/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh Add eth0 [10.133.0.105/23] from ovn-kubernetes 55m Warning ProbeError pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Readiness probe error: Get "http://10.132.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Readiness probe failed: Get "http://10.132.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Normal Started pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Container started 55m Normal SuccessfulCreate replicaset/e2e-mt-fa2c15c1-openshift-default-7d89b995b9 Created pod: e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 55m Normal ScalingReplicaSet deployment/e2e-mt-fa2c15c1-openshift-default Scaled up replica set e2e-mt-fa2c15c1-openshift-default-7d89b995b9 from 0 to 1 55m Normal Created pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Container created 55m Normal AddedLabel gateway/e2e-mt-fa2c15c1 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-fa2c15c1 55m Normal Pulled pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Add eth0 [10.132.0.143/23] from ovn-kubernetes 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-f2bdf3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-f2bdf3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-e166ca-5c477786c9 Error creating: pods "payload-pre-processing-e2e-api-a-e166ca-5c477786c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e166ca: serviceaccount "payload-processing-e2e-api-a-e166ca" not found 55m Warning FailedCreate replicaset/payload-processing-e2e-api-a-e166ca-7974df756c Error creating: pods "payload-processing-e2e-api-a-e166ca-7974df756c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e166ca: serviceaccount "payload-processing-e2e-api-a-e166ca" not found 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-e166ca Scaled up replica set payload-pre-processing-e2e-api-a-e166ca-5c477786c9 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-e166ca-5c477786c9 Created pod: payload-pre-processing-e2e-api-a-e166ca-5c477786c9-79vt2 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-e166ca Scaled up replica set payload-processing-e2e-api-a-e166ca-7974df756c from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-e166ca-7974df756c Created pod: payload-processing-e2e-api-a-e166ca-7974df756c-ltngf 55m Normal Created pod/payload-pre-processing-e2e-api-a-e166ca-5c477786c9-79vt2 Container created 55m Normal Started pod/payload-pre-processing-e2e-api-a-e166ca-5c477786c9-79vt2 Container started 55m Normal AddedInterface pod/payload-processing-e2e-api-a-e166ca-7974df756c-ltngf Add eth0 [10.134.0.139/23] from ovn-kubernetes 55m Normal Pulled pod/payload-pre-processing-e2e-api-a-e166ca-5c477786c9-79vt2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-processing-e2e-api-a-e166ca-7974df756c-ltngf Container started 55m Normal Created pod/payload-processing-e2e-api-a-e166ca-7974df756c-ltngf Container created 55m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-e166ca-5c477786c9-79vt2 Add eth0 [10.134.0.140/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-api-a-e166ca-7974df756c-ltngf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Warning FailedCreate replicaset/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c Error creating: pods "payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-fa2c15c1: serviceaccount "payload-processing-e2e-mt-fa2c15c1" not found 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-fa2c15c1 Scaled up replica set payload-processing-e2e-mt-fa2c15c1-5f4fcb878c from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c Created pod: payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-qrppq 55m Warning Unhealthy pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 55m Warning ProbeError pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc Error creating: pods "payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-fa2c15c1: serviceaccount "payload-processing-e2e-mt-fa2c15c1" not found 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-fa2c15c1 Scaled up replica set payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc Created pod: payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-s9bnm 55m Normal AddedInterface pod/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-qrppq Add eth0 [10.133.0.107/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-s9bnm Add eth0 [10.133.0.106/23] from ovn-kubernetes 55m Normal Pulled pod/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-s9bnm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-qrppq Container created 55m Normal Pulled pod/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-qrppq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-qrppq Container started 55m Normal Started pod/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-s9bnm Container started 55m Normal Created pod/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-s9bnm Container created 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-3c90d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-3c90d8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-3c90d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-3c90d8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Normal AddedLabel gateway/e2e-shared-b-w6-714f8f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-714f8f 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-208e9f32-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-208e9f32-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-714f8f-openshift-default-85496857c7 Created pod: e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh 54m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-714f8f-openshift-default Scaled up replica set e2e-shared-b-w6-714f8f-openshift-default-85496857c7 from 0 to 1 54m Normal AddedInterface pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Add eth0 [10.133.0.108/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687 Created pod: e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-714f8f-85559d789f Error creating: pods "payload-processing-e2e-shared-b-w6-714f8f-85559d789f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-714f8f: serviceaccount "payload-processing-e2e-shared-b-w6-714f8f" not found 54m Normal Pulled pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz Add eth0 [10.132.0.145/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-714f8f Scaled up replica set payload-processing-e2e-shared-b-w6-714f8f-85559d789f from 0 to 1 54m Normal Created pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz Container created 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c Error creating: pods "payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-714f8f: serviceaccount "payload-processing-e2e-shared-b-w6-714f8f" not found 54m Normal Started pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz Container started 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-714f8f Scaled up replica set payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c Created pod: payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-714f8f-85559d789f Created pod: payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x 54m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx Container started 54m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx Container created 54m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x Add eth0 [10.134.0.143/23] from ovn-kubernetes 54m Normal Pulled pod/payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x Container created 54m Normal Started pod/payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx Add eth0 [10.133.0.109/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Container created 54m Warning ProbeError pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Readiness probe error: Get "http://10.134.0.137:15021/healthz/ready": read tcp 10.134.0.2:53180->10.134.0.137:15021: read: connection reset by peer... 54m Normal Started pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Container started 54m Warning Unhealthy pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Readiness probe failed: Get "http://10.134.0.137:15021/healthz/ready": read tcp 10.134.0.2:53180->10.134.0.137:15021: read: connection reset by peer 54m Normal Pulled pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning ProbeError pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz Startup probe error: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz Startup probe failed: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused 54m Warning ProbeError pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused 54m Normal AddedLabel gateway/e2e-api-b-3b1945 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-3b1945 54m Normal ScalingReplicaSet deployment/e2e-api-b-3b1945-openshift-default Scaled up replica set e2e-api-b-3b1945-openshift-default-67fdf96d79 from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-api-b-3b1945-openshift-default-67fdf96d79 Created pod: e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms 54m Normal AddedInterface pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Add eth0 [10.134.0.144/23] from ovn-kubernetes 54m Warning Unhealthy pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Readiness probe failed: Get "http://10.132.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning Unhealthy pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Readiness probe failed: Get "http://10.132.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Readiness probe error: Get "http://10.132.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning ProbeError pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Readiness probe error: Get "http://10.132.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-3b1945-577dddb78f Created pod: payload-processing-e2e-api-b-3b1945-577dddb78f-trzq2 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-3b1945 Scaled up replica set payload-processing-e2e-api-b-3b1945-577dddb78f from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9 Error creating: pods "payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3b1945: serviceaccount "payload-processing-e2e-api-b-3b1945" not found 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9 Created pod: payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-hf7c2 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-3b1945 Scaled up replica set payload-pre-processing-e2e-api-b-3b1945-575c89bbf9 from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-api-b-3b1945-577dddb78f Error creating: pods "payload-processing-e2e-api-b-3b1945-577dddb78f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3b1945: serviceaccount "payload-processing-e2e-api-b-3b1945" not found 54m Normal Created pod/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-hf7c2 Container created 54m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-hf7c2 Add eth0 [10.133.0.110/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-hf7c2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Container created 54m Normal Pulled pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Container started 54m Normal AddedInterface pod/payload-processing-e2e-api-b-3b1945-577dddb78f-trzq2 Add eth0 [10.133.0.111/23] from ovn-kubernetes 54m Normal Started pod/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-hf7c2 Container started 54m Normal Started pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Container started 54m Normal Created pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Container created 54m Normal Pulled pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh Container created 54m Normal Pulled pod/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/e2e-api-b-3d21c7-openshift-default-5dbdb59c99 Created pod: e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 54m Normal AddedLabel gateway/e2e-api-b-3d21c7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-3d21c7 54m Normal ScalingReplicaSet deployment/e2e-api-b-3d21c7-openshift-default Scaled up replica set e2e-api-b-3d21c7-openshift-default-5dbdb59c99 from 0 to 1 54m Normal Started pod/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh Container started 54m Normal Started pod/payload-processing-e2e-api-b-3b1945-577dddb78f-trzq2 Container started 54m Normal Pulled pod/payload-processing-e2e-api-b-3b1945-577dddb78f-trzq2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-api-b-3b1945-577dddb78f-trzq2 Container created 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-bcdb4e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-bcdb4e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal AddedInterface pod/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 Add eth0 [10.132.0.148/23] from ovn-kubernetes 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e166ca-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e166ca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-fa2c15c1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning ProbeError pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Startup probe error: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Startup probe failed: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused 54m Warning ProbeError pod/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-fa2c15c1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-14e039-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-14e039-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning ProbeError pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-3d21c7 Scaled up replica set payload-processing-e2e-api-b-3d21c7-588788f9c from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-3d21c7 Scaled up replica set payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459 from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459 Error creating: pods "payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3d21c7: serviceaccount "payload-processing-e2e-api-b-3d21c7" not found 54m Warning FailedCreate replicaset/payload-processing-e2e-api-b-3d21c7-588788f9c Error creating: pods "payload-processing-e2e-api-b-3d21c7-588788f9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3d21c7: serviceaccount "payload-processing-e2e-api-b-3d21c7" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-3d21c7-588788f9c Created pod: payload-processing-e2e-api-b-3d21c7-588788f9c-h9x65 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459 Created pod: payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-d9gb5 54m Normal AddedInterface pod/payload-processing-e2e-api-b-3d21c7-588788f9c-h9x65 Add eth0 [10.133.0.112/23] from ovn-kubernetes 54m Normal Killing pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 Stopping container payload-pre-processing 54m Normal Killing pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg Stopping container payload-processing 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-d9gb5 Add eth0 [10.133.0.113/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-api-b-3d21c7-588788f9c-h9x65 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-d9gb5 Container started 53m Normal Created pod/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-d9gb5 Container created 53m Normal Created pod/payload-processing-e2e-api-b-3d21c7-588788f9c-h9x65 Container created 53m Normal Started pod/payload-processing-e2e-api-b-3d21c7-588788f9c-h9x65 Container started 53m Normal Pulled pod/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-d9gb5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-14e039-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-14e039-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 Stopping container istio-proxy 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 7; reason: All metrics below target 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Deleted pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-ssq58 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Deleted pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj 53m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-wqdmj Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Deleted pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 53m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-nwkl8 Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled down replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 10 to 7 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 6; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled down replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 10 to 6 53m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled down replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 10 to 6 53m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj 53m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-b7sch Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd 53m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tfzhd Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x 53m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-dclzr Stopping container istio-proxy 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 6; reason: All metrics below target 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-g6tjx 53m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-gk6l2 Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-k6n97 53m Warning Unhealthy pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-hcc54 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 53m Warning Unhealthy pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-vpmkg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 53m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-c6p2x Readiness probe error: HTTP probe failed with statuscode: 503... 53m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-bq6pj Readiness probe error: HTTP probe failed with statuscode: 503... 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e166ca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e166ca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-fa2c15c1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-fa2c15c1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled down replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 10 to 9 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 9; reason: All metrics below target 53m Normal ScalingReplicaSet deployment/e2e-api-a-68f412-openshift-default Scaled up replica set e2e-api-a-68f412-openshift-default-fc857fc67 from 0 to 1 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-xw55s 53m Normal SuccessfulCreate replicaset/e2e-api-a-68f412-openshift-default-fc857fc67 Created pod: e2e-api-a-68f412-openshift-default-fc857fc67-nccrv 53m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-xw55s Stopping container istio-proxy 53m Normal AddedLabel gateway/e2e-api-a-68f412 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-68f412 53m Warning Unhealthy pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Readiness probe failed: Get "http://10.133.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Readiness probe error: Get "http://10.133.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Normal AddedInterface pod/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv Add eth0 [10.133.0.115/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Container created 53m Normal Started pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Container started 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-68f412-9bf56d555 Created pod: payload-pre-processing-e2e-api-a-68f412-9bf56d555-dcm54 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-68f412 Scaled up replica set payload-pre-processing-e2e-api-a-68f412-9bf56d555 from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-api-a-68f412-5c85fb46cc Error creating: pods "payload-processing-e2e-api-a-68f412-5c85fb46cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-68f412: serviceaccount "payload-processing-e2e-api-a-68f412" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-68f412-5c85fb46cc Created pod: payload-processing-e2e-api-a-68f412-5c85fb46cc-nhxzf 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-68f412-9bf56d555 Error creating: pods "payload-pre-processing-e2e-api-a-68f412-9bf56d555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-68f412: serviceaccount "payload-processing-e2e-api-a-68f412" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-68f412 Scaled up replica set payload-processing-e2e-api-a-68f412-5c85fb46cc from 0 to 1 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-68f412-9bf56d555-dcm54 Add eth0 [10.133.0.116/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-api-a-68f412-9bf56d555-dcm54 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-api-a-68f412-9bf56d555-dcm54 Container started 53m Warning Unhealthy pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 53m Normal AddedInterface pod/payload-processing-e2e-api-a-68f412-5c85fb46cc-nhxzf Add eth0 [10.133.0.117/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-api-a-68f412-5c85fb46cc-nhxzf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-api-a-68f412-5c85fb46cc-nhxzf Container created 53m Normal Started pod/payload-processing-e2e-api-a-68f412-5c85fb46cc-nhxzf Container started 53m Normal Created pod/payload-pre-processing-e2e-api-a-68f412-9bf56d555-dcm54 Container created 53m Warning ProbeError pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb 53m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled down replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 9 to 6 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled down replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 10 to 6 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 6; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-mctcb Stopping container istio-proxy 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 6; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 53m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-xgzzp Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-nx8pf 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 53m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-2wn85 Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-jc2v7 Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-tfpjk 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-jgr7l 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-208e9f32 Scaled up replica set payload-processing-e2e-mt-208e9f32-7f6679cdf7 from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-208e9f32 Scaled up replica set payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf Created pod: payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-tlrt5 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf Error creating: pods "payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-208e9f32: serviceaccount "payload-processing-e2e-mt-208e9f32" not found 53m Warning FailedCreate replicaset/payload-processing-e2e-mt-208e9f32-7f6679cdf7 Error creating: pods "payload-processing-e2e-mt-208e9f32-7f6679cdf7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-208e9f32: serviceaccount "payload-processing-e2e-mt-208e9f32" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-208e9f32-7f6679cdf7 Created pod: payload-processing-e2e-mt-208e9f32-7f6679cdf7-jmtjc 53m Normal Pulled pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-jmtjc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-jmtjc Add eth0 [10.133.0.119/23] from ovn-kubernetes 53m Normal Created pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-tlrt5 Container created 53m Normal Pulled pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-tlrt5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-tlrt5 Add eth0 [10.133.0.118/23] from ovn-kubernetes 53m Normal Started pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-tlrt5 Container started 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-714f8f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Created pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-jmtjc Container created 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-714f8f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal Started pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-jmtjc Container started 52m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt Stopping container istio-proxy 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 7; reason: All metrics below target 52m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled down replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 10 to 7 52m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-mzttt 52m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-5pv55 52m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz 52m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-r5tlz Stopping container istio-proxy 52m Normal Started pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Container started 52m Normal Pulled pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Container created 52m Normal Started pod/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 Container started 52m Normal Created pod/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 Container created 52m Normal Killing pod/payload-processing-e2e-api-a-14e039-9bcfbb5f9-2hcl5 Stopping container payload-processing 52m Normal Killing pod/payload-pre-processing-e2e-api-a-14e039-996548779-dvvg4 Stopping container payload-pre-processing 52m Warning ProbeError pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused 52m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 6; reason: All metrics below target 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 6; reason: All metrics below target 52m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh 52m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg 52m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 Stopping container istio-proxy 52m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-hgkfx Stopping container istio-proxy 52m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pjgr9 52m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-d5vwh Stopping container istio-proxy 52m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled down replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 10 to 6 52m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-v82wg Stopping container istio-proxy 52m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled down replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 7 to 6 52m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Readiness probe error: Get "http://10.133.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-r6rs5 Readiness probe failed: Get "http://10.133.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Normal Killing pod/payload-processing-e2e-api-a-e166ca-7974df756c-ltngf Stopping container payload-processing 52m Normal Killing pod/payload-pre-processing-e2e-api-a-e166ca-5c477786c9-79vt2 Stopping container payload-pre-processing 52m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c Stopping container istio-proxy 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 5; reason: All metrics below target 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled down replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 6 to 5 52m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zhd2c 52m Normal Killing pod/payload-pre-processing-e2e-mt-fa2c15c1-5468c658dc-s9bnm Stopping container payload-pre-processing 52m Normal Killing pod/payload-processing-e2e-mt-fa2c15c1-5f4fcb878c-qrppq Stopping container payload-processing 52m Normal SuccessfulCreate replicaset/e2e-inf-a-4d5621-openshift-default-555dc97b75 Created pod: e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht 52m Normal ScalingReplicaSet deployment/e2e-inf-a-4d5621-openshift-default Scaled up replica set e2e-inf-a-4d5621-openshift-default-555dc97b75 from 0 to 1 52m Normal AddedLabel gateway/e2e-inf-a-4d5621 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-4d5621 52m Normal AddedInterface pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Add eth0 [10.133.0.122/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-api-a-bbb902-openshift-default-7c646bcb67 Created pod: e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt 52m Normal AddedLabel gateway/e2e-api-a-bbb902 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-bbb902 52m Normal ScalingReplicaSet deployment/e2e-api-a-bbb902-openshift-default Scaled up replica set e2e-api-a-bbb902-openshift-default-7c646bcb67 from 0 to 1 52m Normal AddedInterface pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Add eth0 [10.133.0.123/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-4d5621 Scaled up replica set payload-processing-e2e-inf-a-4d5621-6747cc9b4f from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-4d5621-6747cc9b4f Created pod: payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs 52m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-4d5621-6747cc9b4f Error creating: pods "payload-processing-e2e-inf-a-4d5621-6747cc9b4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4d5621: serviceaccount "payload-processing-e2e-inf-a-4d5621" not found 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-4d5621 Scaled up replica set payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c Created pod: payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c Error creating: pods "payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4d5621: serviceaccount "payload-processing-e2e-inf-a-4d5621" not found 52m Normal Pulled pod/payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs Container created 52m Normal Started pod/payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs Container started 52m Normal Started pod/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz Container started 52m Normal Pulled pod/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz Add eth0 [10.133.0.124/23] from ovn-kubernetes 52m Normal Created pod/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz Container created 52m Normal AddedInterface pod/payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs Add eth0 [10.133.0.125/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-bbb902 Scaled up replica set payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87 from 0 to 1 52m Warning FailedCreate replicaset/payload-processing-e2e-api-a-bbb902-65d446d445 Error creating: pods "payload-processing-e2e-api-a-bbb902-65d446d445-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-bbb902: serviceaccount "payload-processing-e2e-api-a-bbb902" not found 52m Normal Killing pod/e2e-mt-fa2c15c1-openshift-default-7d89b995b9-829k8 Stopping container istio-proxy 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87 Created pod: payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-69lb6 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-bbb902 Scaled up replica set payload-processing-e2e-api-a-bbb902-65d446d445 from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-bbb902-65d446d445 Created pod: payload-processing-e2e-api-a-bbb902-65d446d445-znrq5 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87 Error creating: pods "payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-bbb902: serviceaccount "payload-processing-e2e-api-a-bbb902" not found 52m Normal Pulled pod/payload-processing-e2e-api-a-bbb902-65d446d445-znrq5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-69lb6 Add eth0 [10.133.0.126/23] from ovn-kubernetes 52m Normal AddedInterface pod/payload-processing-e2e-api-a-bbb902-65d446d445-znrq5 Add eth0 [10.133.0.127/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-69lb6 Container started 52m Normal Created pod/payload-processing-e2e-api-a-bbb902-65d446d445-znrq5 Container created 52m Normal ScalingReplicaSet deployment/e2e-inf-a-fd1dfc-openshift-default Scaled up replica set e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc from 0 to 1 52m Normal Created pod/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-69lb6 Container created 52m Normal SuccessfulCreate replicaset/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc Created pod: e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv 52m Normal AddedLabel gateway/e2e-inf-a-fd1dfc Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-fd1dfc 52m Normal Pulled pod/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-69lb6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Started pod/payload-processing-e2e-api-a-bbb902-65d446d445-znrq5 Container started 52m Normal AddedInterface pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Add eth0 [10.133.0.128/23] from ovn-kubernetes 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3d21c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3d21c7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3b1945-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3b1945-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f Created pod: payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-fd1dfc Scaled up replica set payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-fd1dfc Scaled up replica set payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799 from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799 Created pod: payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 52m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f Error creating: pods "payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-fd1dfc: serviceaccount "payload-processing-e2e-inf-a-fd1dfc" not found 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799 Error creating: pods "payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-fd1dfc: serviceaccount "payload-processing-e2e-inf-a-fd1dfc" not found 52m Normal Created pod/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc Container created 52m Normal Pulled pod/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 Add eth0 [10.133.0.129/23] from ovn-kubernetes 52m Normal Started pod/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc Container started 52m Normal Created pod/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 Container created 52m Normal Pulled pod/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc Add eth0 [10.133.0.130/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 Container started 52m Normal ScalingReplicaSet deployment/e2e-mt-208e9f32-openshift-default Scaled up replica set e2e-mt-208e9f32-openshift-default-77466675fc from 1 to 2 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-208e9f32-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-mt-208e9f32-openshift-default-77466675fc Created pod: e2e-mt-208e9f32-openshift-default-77466675fc-cjpms 52m Normal AddedInterface pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Add eth0 [10.133.0.131/23] from ovn-kubernetes 52m Normal Started pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Container started 52m Normal Created pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Container created 52m Normal Pulled pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedLabel gateway/e2e-api-b-82d887 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-82d887 52m Normal ScalingReplicaSet deployment/e2e-api-b-82d887-openshift-default Scaled up replica set e2e-api-b-82d887-openshift-default-854d6ffc9c from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-api-b-82d887-openshift-default-854d6ffc9c Created pod: e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg 52m Normal AddedInterface pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Add eth0 [10.134.0.146/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv Container created 51m Normal Started pod/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv Container started 51m Warning Unhealthy pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Startup probe failed: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused 51m Warning FailedCreate replicaset/payload-processing-e2e-api-b-82d887-76f55bc4f5 Error creating: pods "payload-processing-e2e-api-b-82d887-76f55bc4f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-82d887: serviceaccount "payload-processing-e2e-api-b-82d887" not found 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-82d887-567965dcbc Error creating: pods "payload-pre-processing-e2e-api-b-82d887-567965dcbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-82d887: serviceaccount "payload-processing-e2e-api-b-82d887" not found 51m Warning ProbeError pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Startup probe error: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused... 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-82d887 Scaled up replica set payload-pre-processing-e2e-api-b-82d887-567965dcbc from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-82d887 Scaled up replica set payload-processing-e2e-api-b-82d887-76f55bc4f5 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-82d887-76f55bc4f5 Created pod: payload-processing-e2e-api-b-82d887-76f55bc4f5-p7m4j 51m Normal Killing pod/e2e-api-a-14e039-openshift-default-5f7fc9955c-pcgls Stopping container istio-proxy 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-82d887-567965dcbc Created pod: payload-pre-processing-e2e-api-b-82d887-567965dcbc-7zpst 51m Normal Pulled pod/payload-pre-processing-e2e-api-b-82d887-567965dcbc-7zpst Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-api-b-82d887-567965dcbc-7zpst Container created 51m Normal Started pod/payload-pre-processing-e2e-api-b-82d887-567965dcbc-7zpst Container started 51m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-82d887-567965dcbc-7zpst Add eth0 [10.133.0.132/23] from ovn-kubernetes 51m Normal Created pod/payload-processing-e2e-api-b-82d887-76f55bc4f5-p7m4j Container created 51m Normal Started pod/payload-processing-e2e-api-b-82d887-76f55bc4f5-p7m4j Container started 51m Normal Pulled pod/payload-processing-e2e-api-b-82d887-76f55bc4f5-p7m4j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-api-b-82d887-76f55bc4f5-p7m4j Add eth0 [10.133.0.133/23] from ovn-kubernetes 51m Warning ProbeError pod/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused 51m Normal Killing pod/payload-processing-e2e-api-b-3b1945-577dddb78f-trzq2 Stopping container payload-processing 51m Normal Killing pod/payload-pre-processing-e2e-api-b-3b1945-575c89bbf9-hf7c2 Stopping container payload-pre-processing 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-68f412-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-68f412-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Killing pod/e2e-api-a-e166ca-openshift-default-6584656577-7cxfh Stopping container istio-proxy 51m Normal Killing pod/payload-pre-processing-e2e-mt-208e9f32-5b998b4dcf-tlrt5 Stopping container payload-pre-processing 51m Normal Killing pod/payload-processing-e2e-mt-208e9f32-7f6679cdf7-jmtjc Stopping container payload-processing 51m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth Stopping container istio-proxy 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 5; reason: All metrics below target 51m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt 51m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-wzcwt Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled down replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 7 to 5 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 5; reason: All metrics below target 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 5; reason: All metrics below target 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled down replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 6 to 5 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Deleted pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7whth 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Deleted pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b 51m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-rdw5b Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default Scaled down replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 6 to 5 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 5; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw 51m Normal Killing pod/payload-processing-e2e-api-b-3d21c7-588788f9c-h9x65 Stopping container payload-processing 51m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-kqlkn 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled down replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 6 to 5 51m Normal Killing pod/payload-pre-processing-e2e-api-b-3d21c7-65ffb6f459-d9gb5 Stopping container payload-pre-processing 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 5; reason: All metrics below target 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled down replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 6 to 5 51m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-48jkw Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled down replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 6 to 5 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-68f412-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-68f412-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-ln876 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 5; reason: All metrics below target 51m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Readiness probe failed: HTTP probe failed with statuscode: 503 51m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-nn2c6 Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal SuccessfulCreate replicaset/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9 Created pod: e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d 50m Normal AddedLabel gateway/e2e-inf-b-c6d97d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-c6d97d 50m Normal ScalingReplicaSet deployment/e2e-inf-b-c6d97d-openshift-default Scaled up replica set e2e-inf-b-c6d97d-openshift-default-79fc67f7c9 from 0 to 1 50m Normal AddedInterface pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Add eth0 [10.133.0.137/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/e2e-api-b-bdb61f-openshift-default-75c44c9f85 Created pod: e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq 50m Normal ScalingReplicaSet deployment/e2e-api-b-bdb61f-openshift-default Scaled up replica set e2e-api-b-bdb61f-openshift-default-75c44c9f85 from 0 to 1 50m Normal AddedLabel gateway/e2e-api-b-bdb61f Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-bdb61f 50m Warning Unhealthy pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Readiness probe failed: Get "http://10.133.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Readiness probe error: Get "http://10.133.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal AddedInterface pod/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq Add eth0 [10.134.0.148/23] from ovn-kubernetes 50m Normal Started pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Container started 50m Normal Pulled pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Container created 50m Normal Started pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Container started 50m Normal Pulled pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Container created 50m Warning ProbeError pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Readiness probe error: Get "http://10.133.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-c6d97d Scaled up replica set payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55 Created pod: payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55 Error creating: pods "payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c6d97d: serviceaccount "payload-processing-e2e-inf-b-c6d97d" not found 50m Warning Unhealthy pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Readiness probe failed: Get "http://10.133.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-c6d97d Scaled up replica set payload-processing-e2e-inf-b-c6d97d-68bd6f89 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-c6d97d-68bd6f89 Created pod: payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-c6d97d-68bd6f89 Error creating: pods "payload-processing-e2e-inf-b-c6d97d-68bd6f89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c6d97d: serviceaccount "payload-processing-e2e-inf-b-c6d97d" not found 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 Add eth0 [10.133.0.138/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd Add eth0 [10.133.0.139/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Pulled pod/payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-f2bdf3-openshift-default New size: 2; reason: All metrics below target 50m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-f2bdf3-openshift-default Scaled down replica set e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 from 3 to 2 50m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 Deleted pod: e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd 50m Normal Started pod/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 Container started 50m Normal Created pod/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 Container created 50m Normal Killing pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-2b2nd Stopping container istio-proxy 50m Normal Created pod/payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd Container created 50m Normal Started pod/payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd Container started 50m Warning ProbeError pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-bdb61f-764449d9c5 Created pod: payload-processing-e2e-api-b-bdb61f-764449d9c5-86l2g 50m Normal AddedLabel gateway/e2e-inf-b-609843 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-609843 50m Warning FailedCreate replicaset/payload-processing-e2e-api-b-bdb61f-764449d9c5 Error creating: pods "payload-processing-e2e-api-b-bdb61f-764449d9c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bdb61f: serviceaccount "payload-processing-e2e-api-b-bdb61f" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c Created pod: payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-tw6mt 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-bdb61f Scaled up replica set payload-processing-e2e-api-b-bdb61f-764449d9c5 from 0 to 1 50m Normal ScalingReplicaSet deployment/e2e-inf-b-609843-openshift-default Scaled up replica set e2e-inf-b-609843-openshift-default-db7868f45 from 0 to 1 50m Normal Pulled pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/e2e-inf-b-609843-openshift-default-db7868f45 Created pod: e2e-inf-b-609843-openshift-default-db7868f45-jmbpz 50m Normal Created pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Container created 50m Warning Unhealthy pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-bdb61f Scaled up replica set payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c Error creating: pods "payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bdb61f: serviceaccount "payload-processing-e2e-api-b-bdb61f" not found 50m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-tw6mt Add eth0 [10.133.0.140/23] from ovn-kubernetes 50m Normal Started pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Container started 50m Normal Started pod/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-tw6mt Container started 50m Normal Created pod/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-tw6mt Container created 50m Normal Pulled pod/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-tw6mt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-api-b-bdb61f-764449d9c5-86l2g Add eth0 [10.134.0.149/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-api-b-bdb61f-764449d9c5-86l2g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-api-b-bdb61f-764449d9c5-86l2g Container created 50m Normal Started pod/payload-processing-e2e-api-b-bdb61f-764449d9c5-86l2g Container started 50m Warning Unhealthy pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Startup probe failed: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused 50m Normal AddedInterface pod/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz Add eth0 [10.132.0.159/23] from ovn-kubernetes 50m Warning ProbeError pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Startup probe error: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused... 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-fd1dfc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-fd1dfc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-609843 Scaled up replica set payload-pre-processing-e2e-inf-b-609843-79599884c from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-609843-b8d75c975 Error creating: pods "payload-processing-e2e-inf-b-609843-b8d75c975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-609843: serviceaccount "payload-processing-e2e-inf-b-609843" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-609843 Scaled up replica set payload-processing-e2e-inf-b-609843-b8d75c975 from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-609843-79599884c Error creating: pods "payload-pre-processing-e2e-inf-b-609843-79599884c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-609843: serviceaccount "payload-processing-e2e-inf-b-609843" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-609843-79599884c Created pod: payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-609843-b8d75c975 Created pod: payload-processing-e2e-inf-b-609843-b8d75c975-frvqn 50m Warning ProbeError pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t Add eth0 [10.133.0.141/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t Container started 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4d5621-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Created pod/payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t Container created 50m Normal Started pod/payload-processing-e2e-inf-b-609843-b8d75c975-frvqn Container started 50m Normal Created pod/payload-processing-e2e-inf-b-609843-b8d75c975-frvqn Container created 50m Normal Pulled pod/payload-processing-e2e-inf-b-609843-b8d75c975-frvqn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-inf-b-609843-b8d75c975-frvqn Add eth0 [10.133.0.142/23] from ovn-kubernetes 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-bbb902-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-bbb902-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4d5621-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Killing pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Stopping container istio-proxy 50m Normal Killing pod/e2e-mt-208e9f32-openshift-default-77466675fc-qqt97 Stopping container istio-proxy 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-fd1dfc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-fd1dfc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning Unhealthy pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-mt-208e9f32-openshift-default-77466675fc-cjpms Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal Killing pod/payload-pre-processing-e2e-api-a-68f412-9bf56d555-dcm54 Stopping container payload-pre-processing 50m Normal Killing pod/payload-processing-e2e-api-a-68f412-5c85fb46cc-nhxzf Stopping container payload-processing 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-bbb902-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-bbb902-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Killing pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Stopping container istio-proxy 50m Normal SuccessfulCreate replicaset/e2e-api-b-82d887-openshift-default-854d6ffc9c Created pod: e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-82d887-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-api-b-82d887-openshift-default-854d6ffc9c Created pod: e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m 50m Normal SuccessfulCreate replicaset/e2e-api-b-82d887-openshift-default-854d6ffc9c Created pod: e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck 50m Normal ScalingReplicaSet deployment/e2e-api-b-82d887-openshift-default Scaled up replica set e2e-api-b-82d887-openshift-default-854d6ffc9c from 1 to 4 50m Normal Pulled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck Container started 50m Normal AddedInterface pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp Add eth0 [10.133.0.143/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/e2e-inf-a-b87c62-openshift-default-79f485ccf5 Created pod: e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk 50m Normal Created pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m Container created 50m Normal AddedInterface pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m Add eth0 [10.134.0.150/23] from ovn-kubernetes 50m Normal Started pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp Container started 50m Normal Pulled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m Container started 50m Normal AddedLabel gateway/e2e-inf-a-b87c62 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-b87c62 50m Normal Created pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck Container created 50m Normal Pulled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp Container created 50m Normal ScalingReplicaSet deployment/e2e-inf-a-b87c62-openshift-default Scaled up replica set e2e-inf-a-b87c62-openshift-default-79f485ccf5 from 0 to 1 50m Normal AddedInterface pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck Add eth0 [10.132.0.161/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Add eth0 [10.133.0.144/23] from ovn-kubernetes 50m Warning Unhealthy pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-api-b-3b1945-openshift-default-67fdf96d79-5jpms Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Readiness probe error: Get "http://10.134.0.146:15021/healthz/ready": read tcp 10.134.0.2:50868->10.134.0.146:15021: read: connection reset by peer... 50m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Readiness probe failed: Get "http://10.134.0.146:15021/healthz/ready": read tcp 10.134.0.2:50868->10.134.0.146:15021: read: connection reset by peer 50m Normal Created pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Container created 50m Normal Killing pod/e2e-api-b-3d21c7-openshift-default-5dbdb59c99-gz7q4 Stopping container istio-proxy 50m Normal Started pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Container started 50m Normal Pulled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedLabel gateway/e2e-inf-a-762753 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-762753 50m Normal ScalingReplicaSet deployment/e2e-inf-a-762753-openshift-default Scaled up replica set e2e-inf-a-762753-openshift-default-85767dbb67 from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-inf-a-762753-openshift-default-85767dbb67 Created pod: e2e-inf-a-762753-openshift-default-85767dbb67-8dghm 50m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 50m Normal AddedInterface pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Add eth0 [10.134.0.151/23] from ovn-kubernetes 50m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m Startup probe failed: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused 50m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 50m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m Startup probe error: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-b87c62-6b8656d99c Error creating: pods "payload-processing-e2e-inf-a-b87c62-6b8656d99c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b87c62: serviceaccount "payload-processing-e2e-inf-a-b87c62" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-b87c62 Scaled up replica set payload-processing-e2e-inf-a-b87c62-6b8656d99c from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775 Error creating: pods "payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b87c62: serviceaccount "payload-processing-e2e-inf-a-b87c62" not found 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4d5621-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775 Created pod: payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-b87c62-6b8656d99c Created pod: payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-b87c62 Scaled up replica set payload-pre-processing-e2e-inf-a-b87c62-7745bdc775 from 0 to 1 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4d5621-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Started pod/payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 Container started 50m Normal Created pod/payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 Container created 50m Normal Pulled pod/payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b Add eth0 [10.133.0.145/23] from ovn-kubernetes 50m Normal Started pod/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b Container started 50m Normal AddedInterface pod/payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 Add eth0 [10.133.0.146/23] from ovn-kubernetes 50m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 50m Normal Pulled pod/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b Container created 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-762753-64f6f4597c Created pod: payload-processing-e2e-inf-a-762753-64f6f4597c-vnz4m 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f Created pod: payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-762753-64f6f4597c Error creating: pods "payload-processing-e2e-inf-a-762753-64f6f4597c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-762753: serviceaccount "payload-processing-e2e-inf-a-762753" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-762753 Scaled up replica set payload-processing-e2e-inf-a-762753-64f6f4597c from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-762753 Scaled up replica set payload-pre-processing-e2e-inf-a-762753-6847dc7c9f from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f Error creating: pods "payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-762753: serviceaccount "payload-processing-e2e-inf-a-762753" not found 50m Normal Pulled pod/payload-processing-e2e-inf-a-762753-64f6f4597c-vnz4m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-inf-a-762753-64f6f4597c-vnz4m Container created 50m Normal Started pod/payload-processing-e2e-inf-a-762753-64f6f4597c-vnz4m Container started 50m Normal AddedInterface pod/payload-processing-e2e-inf-a-762753-64f6f4597c-vnz4m Add eth0 [10.134.0.153/23] from ovn-kubernetes 50m Normal Started pod/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs Container started 50m Normal Created pod/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs Container created 50m Normal Pulled pod/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs Add eth0 [10.134.0.152/23] from ovn-kubernetes 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-82d887-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-82d887-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-82d887-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-82d887-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b87c62-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b87c62-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687 Deleted pod: e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz 49m Normal Killing pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b22gz Stopping container istio-proxy 49m Normal SuccessfulCreate replicaset/e2e-api-b-82d887-openshift-default-854d6ffc9c Created pod: e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm 49m Normal Pulled pod/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-api-b-82d887-openshift-default-854d6ffc9c Created pod: e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 49m Normal Created pod/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz Container created 49m Normal SuccessfulCreate replicaset/e2e-api-b-82d887-openshift-default-854d6ffc9c Created pod: e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz 49m Normal Started pod/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz Container started 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-82d887-openshift-default New size: 7; reason: All metrics below target 49m Normal ScalingReplicaSet deployment/e2e-api-b-82d887-openshift-default Scaled up replica set e2e-api-b-82d887-openshift-default-854d6ffc9c from 4 to 7 49m Normal Created pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 Container created 49m Normal Created pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm Container created 49m Normal Started pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz Container started 49m Normal Created pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz Container created 49m Normal Pulled pod/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz Add eth0 [10.134.0.154/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm Add eth0 [10.132.0.166/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm Container started 49m Normal Created pod/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq Container created 49m Normal Pulled pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 Add eth0 [10.133.0.147/23] from ovn-kubernetes 49m Normal Started pod/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq Container started 49m Normal Started pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 Container started 49m Warning Unhealthy pod/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bdb61f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bdb61f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm Startup probe failed: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 Startup probe error: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm Startup probe error: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 Startup probe failed: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Readiness probe failed: Get "http://10.133.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Readiness probe error: Get "http://10.133.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal Killing pod/e2e-api-a-68f412-openshift-default-fc857fc67-nccrv Stopping container istio-proxy 49m Normal Pulled pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Container created 49m Normal Started pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Container started 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-609843-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-609843-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal Killing pod/payload-pre-processing-e2e-api-a-bbb902-74dbf5bf87-69lb6 Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-api-a-bbb902-65d446d445-znrq5 Stopping container payload-processing 49m Warning ProbeError pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Startup probe error: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Startup probe failed: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c6d97d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c6d97d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Killing pod/payload-pre-processing-e2e-api-b-82d887-567965dcbc-7zpst Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-api-b-82d887-76f55bc4f5-p7m4j Stopping container payload-processing 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-609843-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-609843-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bdb61f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bdb61f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default New size: 4; reason: All metrics below target 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled down replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 5 to 4 48m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Stopping container istio-proxy 48m Normal SuccessfulCreate replicaset/e2e-inf-b-01a117-openshift-default-56c5c46dc5 Created pod: e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz 48m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 Stopping container istio-proxy 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 4; reason: All metrics below target 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled down replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 5 to 4 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 4; reason: All metrics below target 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 4; reason: All metrics below target 48m Normal AddedLabel gateway/e2e-inf-b-01a117 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-01a117 48m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr 48m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 48m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-z5s42 Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm 48m Normal ScalingReplicaSet deployment/e2e-inf-b-01a117-openshift-default Scaled up replica set e2e-inf-b-01a117-openshift-default-56c5c46dc5 from 0 to 1 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled down replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 5 to 4 48m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 48m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-t9ds7 Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-vjpz9 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 4; reason: All metrics below target 48m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default Scaled down replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 5 to 4 48m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-kzfbr Stopping container istio-proxy 48m Normal AddedInterface pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Add eth0 [10.133.0.151/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Readiness probe failed: Get "http://10.133.0.144:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Readiness probe error: Get "http://10.133.0.144:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c6d97d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c6d97d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-nc7dm Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal Pulled pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning ProbeError pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Readiness probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 48m Normal Created pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Container created 48m Normal AddedLabel gateway/e2e-inf-b-9de7f5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-9de7f5 48m Normal ScalingReplicaSet deployment/e2e-inf-b-9de7f5-openshift-default Scaled up replica set e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f from 0 to 1 48m Normal Started pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Container started 48m Warning Unhealthy pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Readiness probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 48m Normal SuccessfulCreate replicaset/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f Created pod: e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt 48m Normal Created pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Container created 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-01a117 Scaled up replica set payload-processing-e2e-inf-b-01a117-849b4fc645 from 0 to 1 48m Normal Started pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Container started 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99 Created pod: payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 48m Normal Pulled pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-01a117-849b4fc645 Error creating: pods "payload-processing-e2e-inf-b-01a117-849b4fc645-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-01a117: serviceaccount "payload-processing-e2e-inf-b-01a117" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-01a117-849b4fc645 Created pod: payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf 48m Normal AddedInterface pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt Add eth0 [10.133.0.152/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-01a117 Scaled up replica set payload-pre-processing-e2e-inf-b-01a117-96d7f8b99 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99 Error creating: pods "payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-01a117: serviceaccount "payload-processing-e2e-inf-b-01a117" not found 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 Add eth0 [10.133.0.153/23] from ovn-kubernetes 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf Add eth0 [10.133.0.154/23] from ovn-kubernetes 48m Normal Started pod/payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf Container started 48m Normal Started pod/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 Container started 48m Normal Pulled pod/payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 Container created 48m Normal Created pod/payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf Container created 48m Normal ScalingReplicaSet deployment/e2e-inf-a-72120c-openshift-default Scaled up replica set e2e-inf-a-72120c-openshift-default-77d78ddc45 from 0 to 1 48m Normal AddedLabel gateway/e2e-inf-a-72120c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-72120c 48m Normal SuccessfulCreate replicaset/e2e-inf-a-72120c-openshift-default-77d78ddc45 Created pod: e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-9de7f5 Scaled up replica set payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7 from 0 to 1 48m Warning ProbeError pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 48m Normal AddedInterface pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Add eth0 [10.132.0.171/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-9de7f5 Scaled up replica set payload-processing-e2e-inf-b-9de7f5-6b8cc55f75 from 0 to 1 48m Warning ProbeError pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7 Created pod: payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7 Error creating: pods "payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-9de7f5: serviceaccount "payload-processing-e2e-inf-b-9de7f5" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75 Error creating: pods "payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-9de7f5: serviceaccount "payload-processing-e2e-inf-b-9de7f5" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75 Created pod: payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj 48m Warning Unhealthy pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm Add eth0 [10.133.0.155/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm Container started 48m Normal Created pod/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj Container created 48m Normal Pulled pod/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj Add eth0 [10.134.0.156/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj Container started 48m Normal Created pod/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm Container created 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-72120c Scaled up replica set payload-processing-e2e-inf-a-72120c-5d4968bd99 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-72120c Scaled up replica set payload-pre-processing-e2e-inf-a-72120c-7776cf45f from 0 to 1 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-762753-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-72120c-7776cf45f Error creating: pods "payload-pre-processing-e2e-inf-a-72120c-7776cf45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-72120c: serviceaccount "payload-processing-e2e-inf-a-72120c" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-72120c-5d4968bd99 Error creating: pods "payload-processing-e2e-inf-a-72120c-5d4968bd99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-72120c: serviceaccount "payload-processing-e2e-inf-a-72120c" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-72120c-5d4968bd99 Created pod: payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-72120c-7776cf45f Created pod: payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-762753-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal Pulled pod/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 Container started 48m Normal Created pod/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd Container created 48m Normal AddedInterface pod/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd Add eth0 [10.133.0.157/23] from ovn-kubernetes 48m Normal Started pod/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 Add eth0 [10.133.0.156/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 Container created 48m Normal ScalingReplicaSet deployment/e2e-inf-a-4d5621-openshift-default Scaled up replica set e2e-inf-a-4d5621-openshift-default-555dc97b75 from 1 to 2 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-4d5621-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-inf-a-4d5621-openshift-default-555dc97b75 Created pod: e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 48m Normal Started pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 Container started 48m Normal AddedInterface pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 Add eth0 [10.132.0.172/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 Container created 48m Warning ProbeError pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 Startup probe error: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 Startup probe failed: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default New size: 3; reason: All metrics below target 48m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-zv6bb 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-762753-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-762753-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b87c62-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b87c62-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Killing pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Stopping container istio-proxy 47m Warning ProbeError pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-api-a-bbb902-openshift-default-7c646bcb67-wkvqt Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal Killing pod/payload-processing-e2e-api-b-bdb61f-764449d9c5-86l2g Stopping container payload-processing 47m Normal Killing pod/payload-pre-processing-e2e-api-b-bdb61f-69cd5d978c-tw6mt Stopping container payload-pre-processing 47m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled down replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 5 to 4 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 4; reason: All metrics below target 47m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln 47m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Readiness probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-fjjln Readiness probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 47m Normal Killing pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-dxv97 Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Stopping container istio-proxy 47m Warning ProbeError pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal Killing pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jgjck Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jt2rz Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-6268m Stopping container istio-proxy 47m Normal Killing pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jz7xm Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-brrdg Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal Killing pod/e2e-api-b-82d887-openshift-default-854d6ffc9c-jdktp Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-inf-a-57e594-openshift-default Scaled up replica set e2e-inf-a-57e594-openshift-default-55cf466c9c from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-inf-a-57e594-openshift-default-55cf466c9c Created pod: e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 47m Normal AddedLabel gateway/e2e-inf-a-57e594 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-57e594 47m Normal AddedInterface pod/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 Add eth0 [10.133.0.159/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c Created pod: payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c Error creating: pods "payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-57e594: serviceaccount "payload-processing-e2e-inf-a-57e594" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-57e594 Scaled up replica set payload-processing-e2e-inf-a-57e594-86cdf6574c from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-57e594 Scaled up replica set payload-pre-processing-e2e-inf-a-57e594-6784f95c8c from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-57e594-86cdf6574c Created pod: payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-57e594-86cdf6574c Error creating: pods "payload-processing-e2e-inf-a-57e594-86cdf6574c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-57e594: serviceaccount "payload-processing-e2e-inf-a-57e594" not found 47m Normal Created pod/payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 Container created 47m Normal Started pod/payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 Container started 47m Normal Started pod/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 Container started 47m Normal Pulled pod/payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 Container created 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 Add eth0 [10.133.0.161/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 Add eth0 [10.133.0.160/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/e2e-inf-b-82b5e8-openshift-default Scaled up replica set e2e-inf-b-82b5e8-openshift-default-79b8ff867c from 0 to 1 46m Normal AddedLabel gateway/e2e-inf-b-82b5e8 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-82b5e8 46m Normal SuccessfulCreate replicaset/e2e-inf-b-82b5e8-openshift-default-79b8ff867c Created pod: e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt 46m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 4; reason: All metrics below target 46m Normal AddedInterface pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt Add eth0 [10.134.0.159/23] from ovn-kubernetes 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Deleted pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-tftpg 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default Scaled down replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 5 to 4 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc Created pod: payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf Error creating: pods "payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-82b5e8: serviceaccount "payload-processing-e2e-inf-b-82b5e8" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc Error creating: pods "payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-82b5e8: serviceaccount "payload-processing-e2e-inf-b-82b5e8" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf Created pod: payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-4z7ng 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-82b5e8 Scaled up replica set payload-processing-e2e-inf-b-82b5e8-5cc6c765cf from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-82b5e8 Scaled up replica set payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc from 0 to 1 46m Normal Started pod/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-4z7ng Container started 46m Normal Created pod/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-4z7ng Container created 46m Normal Pulled pod/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-4z7ng Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-4z7ng Add eth0 [10.133.0.162/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 Add eth0 [10.133.0.163/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 Container started 46m Normal Created pod/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 Container created 46m Normal Pulled pod/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Warning ProbeError pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Readiness probe error: Get "http://10.132.0.171:15021/healthz/ready": read tcp 10.132.0.2:36218->10.132.0.171:15021: read: connection reset by peer... 46m Warning Unhealthy pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Readiness probe failed: Get "http://10.132.0.171:15021/healthz/ready": read tcp 10.132.0.2:36218->10.132.0.171:15021: read: connection reset by peer 46m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Stopping container istio-proxy 46m Normal Pulled pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 46m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 3; reason: All metrics below target 46m Normal Pulled pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt Container started 46m Normal Created pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Container created 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-xtn2l 46m Normal Started pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Container started 46m Normal Created pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt Container created 46m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default Scaled down replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 4 to 3 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw 46m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-7nldw Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-knx22 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default Scaled down replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 4 to 3 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default Scaled down replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 4 to 3 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 3; reason: All metrics below target 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 3; reason: All metrics below target 46m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-t7kb5 Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning Unhealthy pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-72120c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-72120c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-82b5e8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-82b5e8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-9de7f5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-9de7f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Killing pod/e2e-api-b-bdb61f-openshift-default-75c44c9f85-pq6sq Stopping container istio-proxy 46m Normal SuccessfulCreate replicaset/e2e-inf-a-c942c9-openshift-default-5bdd576867 Created pod: e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 46m Normal AddedLabel gateway/e2e-inf-a-c942c9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c942c9 46m Normal ScalingReplicaSet deployment/e2e-inf-a-c942c9-openshift-default Scaled up replica set e2e-inf-a-c942c9-openshift-default-5bdd576867 from 0 to 1 46m Normal AddedInterface pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Add eth0 [10.134.0.160/23] from ovn-kubernetes 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-72120c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-72120c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f Created pod: payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c942c9 Scaled up replica set payload-pre-processing-e2e-inf-a-c942c9-6f77df795f from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c942c9-68f5b49c89 Error creating: pods "payload-processing-e2e-inf-a-c942c9-68f5b49c89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c942c9: serviceaccount "payload-processing-e2e-inf-a-c942c9" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c942c9 Scaled up replica set payload-processing-e2e-inf-a-c942c9-68f5b49c89 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c942c9-68f5b49c89 Created pod: payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f Error creating: pods "payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c942c9: serviceaccount "payload-processing-e2e-inf-a-c942c9" not found 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk Add eth0 [10.134.0.161/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k Container started 46m Normal Started pod/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk Container started 46m Normal AddedInterface pod/payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k Add eth0 [10.134.0.162/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk Container created 46m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k Container created 46m Normal Pulled pod/payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 Container created 46m Normal Pulled pod/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 Container started 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-9de7f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-9de7f5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-57e594-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-57e594-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning Unhealthy pod/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 Startup probe failed: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused 46m Warning ProbeError pod/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 Startup probe error: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused... 45m Normal AddedLabel gateway/e2e-inf-b-4f4e69 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-4f4e69 45m Normal ScalingReplicaSet deployment/e2e-inf-b-4f4e69-openshift-default Scaled up replica set e2e-inf-b-4f4e69-openshift-default-58bcbfbb57 from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57 Created pod: e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t 45m Normal AddedInterface pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Add eth0 [10.134.0.163/23] from ovn-kubernetes 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-4f4e69-5f8fb86855 Error creating: pods "payload-processing-e2e-inf-b-4f4e69-5f8fb86855-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4f4e69: serviceaccount "payload-processing-e2e-inf-b-4f4e69" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-4f4e69 Scaled up replica set payload-processing-e2e-inf-b-4f4e69-5f8fb86855 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-4f4e69 Scaled up replica set payload-pre-processing-e2e-inf-b-4f4e69-579b866c9 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9 Created pod: payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9 Error creating: pods "payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4f4e69: serviceaccount "payload-processing-e2e-inf-b-4f4e69" not found 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 Add eth0 [10.133.0.164/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 Container created 45m Normal Started pod/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 Container started 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-4f4e69-5f8fb86855 Created pod: payload-processing-e2e-inf-b-4f4e69-5f8fb86855-mzzfn 45m Normal Started pod/payload-processing-e2e-inf-b-4f4e69-5f8fb86855-mzzfn Container started 45m Normal Created pod/payload-processing-e2e-inf-b-4f4e69-5f8fb86855-mzzfn Container created 45m Normal Pulled pod/payload-processing-e2e-inf-b-4f4e69-5f8fb86855-mzzfn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-4f4e69-5f8fb86855-mzzfn Add eth0 [10.132.0.180/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-57e594-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-57e594-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Readiness probe error: Get "http://10.133.0.151:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Readiness probe failed: Get "http://10.133.0.151:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal Pulled pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Container created 45m Normal Started pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Container started 45m Normal Created pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt Container created 45m Normal Pulled pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt Container started 45m Warning ProbeError pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Startup probe error: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Startup probe failed: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt Startup probe failed: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused 45m Warning ProbeError pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt Startup probe error: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused... 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-01a117-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-01a117-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal ScalingReplicaSet deployment/e2e-inf-b-9de7f5-openshift-default Scaled up replica set e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f from 1 to 2 45m Normal SuccessfulCreate replicaset/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f Created pod: e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv 45m Normal Pulled pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Add eth0 [10.132.0.182/23] from ovn-kubernetes 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-9de7f5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Normal Created pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Container created 45m Normal Started pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Container started 44m Warning ProbeError pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Startup probe error: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Startup probe failed: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 3; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p 44m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4k69p Stopping container istio-proxy 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-01a117-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-01a117-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-82b5e8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-82b5e8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal AddedLabel gateway/e2e-inf-b-6f8856 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-6f8856 44m Normal Killing pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-inf-b-6f8856-openshift-default Scaled up replica set e2e-inf-b-6f8856-openshift-default-6f9ff5d847 from 0 to 1 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-36af31-openshift-default Scaled down replica set e2e-shared-a-w2-36af31-openshift-default-74f58f595f from 2 to 1 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-36af31-openshift-default New size: 1; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-shared-a-w2-36af31-openshift-default-74f58f595f Deleted pod: e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp 44m Normal SuccessfulCreate replicaset/e2e-inf-b-6f8856-openshift-default-6f9ff5d847 Created pod: e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt 44m Normal AddedInterface pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Add eth0 [10.134.0.165/23] from ovn-kubernetes 44m Warning Unhealthy pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-b4qbp Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-6f8856 Scaled up replica set payload-processing-e2e-inf-b-6f8856-647b85df9c from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-6f8856-9c858b495 Error creating: pods "payload-pre-processing-e2e-inf-b-6f8856-9c858b495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-6f8856: serviceaccount "payload-processing-e2e-inf-b-6f8856" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-6f8856-9c858b495 Created pod: payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-6f8856 Scaled up replica set payload-pre-processing-e2e-inf-b-6f8856-9c858b495 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-6f8856-647b85df9c Error creating: pods "payload-processing-e2e-inf-b-6f8856-647b85df9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-6f8856: serviceaccount "payload-processing-e2e-inf-b-6f8856" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-6f8856-647b85df9c Created pod: payload-processing-e2e-inf-b-6f8856-647b85df9c-tbspl 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt Add eth0 [10.133.0.167/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt Container created 44m Normal Started pod/payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt Container started 44m Normal Pulled pod/payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-processing-e2e-inf-b-6f8856-647b85df9c-tbspl Container started 44m Normal Created pod/payload-processing-e2e-inf-b-6f8856-647b85df9c-tbspl Container created 44m Normal Pulled pod/payload-processing-e2e-inf-b-6f8856-647b85df9c-tbspl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-inf-b-6f8856-647b85df9c-tbspl Add eth0 [10.133.0.168/23] from ovn-kubernetes 44m Normal Pulled pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Container started 44m Normal Created pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Container created 44m Warning Unhealthy pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Readiness probe failed: Get "http://10.134.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Readiness probe error: Get "http://10.134.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Normal Pulled pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Container started 44m Normal Created pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Container created 44m Warning ProbeError pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Startup probe error: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Startup probe failed: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Startup probe failed: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused 44m Warning ProbeError pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Startup probe error: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused... 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4f4e69-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4f4e69-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-6f8856-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-6f8856-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c942c9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c942c9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4f4e69-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4f4e69-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c942c9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c942c9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal ScalingReplicaSet deployment/e2e-inf-b-82b5e8-openshift-default Scaled up replica set e2e-inf-b-82b5e8-openshift-default-79b8ff867c from 1 to 2 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-82b5e8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/e2e-inf-b-82b5e8-openshift-default-79b8ff867c Created pod: e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 43m Normal Pulled pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 Add eth0 [10.133.0.178/23] from ovn-kubernetes 43m Normal Started pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 Container started 43m Normal Created pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 Container created 43m Normal SuccessfulCreate replicaset/e2e-inf-a-762753-openshift-default-85767dbb67 Created pod: e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-762753-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/e2e-inf-a-762753-openshift-default Scaled up replica set e2e-inf-a-762753-openshift-default-85767dbb67 from 1 to 2 43m Warning Unhealthy pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 Startup probe failed: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused 43m Normal Created pod/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg Container created 43m Normal Started pod/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg Container started 43m Normal Pulled pod/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning ProbeError pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 Startup probe error: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused... 43m Normal AddedInterface pod/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg Add eth0 [10.132.0.194/23] from ovn-kubernetes 43m Warning ProbeError pod/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg Startup probe error: Get "http://10.132.0.194:15021/healthz/ready": dial tcp 10.132.0.194:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg Startup probe failed: Get "http://10.132.0.194:15021/healthz/ready": dial tcp 10.132.0.194:15021: connect: connection refused 43m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Deleted pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 3; reason: All metrics below target 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default New size: 2; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-55w4l 43m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh 43m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-55w4l Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-lhgqf 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default New size: 2; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default New size: 2; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-wmmjm 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz 43m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Stopping container istio-proxy 43m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-dldvh Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Readiness probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 43m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-db847 Readiness probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-7rfqz Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal Killing pod/payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz Stopping container payload-pre-processing 42m Warning Unhealthy pod/payload-processing-e2e-inf-a-4d5621-6747cc9b4f-7ncgs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-4d5621-5c496fd68c-mmnpz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Normal Killing pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-gcch2 Stopping container istio-proxy 42m Normal Killing pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Stopping container istio-proxy 42m Warning ProbeError pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-inf-a-4d5621-openshift-default-555dc97b75-zpbht Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal Started pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Container started 42m Normal Created pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Container created 42m Normal Pulled pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning Unhealthy pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Startup probe failed: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Startup probe error: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused... 42m Normal Killing pod/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 Stopping container payload-pre-processing 42m Normal Killing pod/payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd Stopping container payload-processing 42m Normal Killing pod/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 Stopping container payload-pre-processing 42m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-8zxnz Stopping container istio-proxy 42m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Stopping container istio-proxy 42m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-8zxnz 42m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6886b96f9 from 10 to 7 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-hk9qf 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-4qhrd 42m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-4qhrd Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-hk9qf Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning Unhealthy pod/payload-processing-e2e-inf-b-c6d97d-68bd6f89-nwznd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-c6d97d-59565cfc55-pkzq9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-fd1dfc-5fc4c54799-65s28 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Warning Unhealthy pod/payload-processing-e2e-inf-a-fd1dfc-6b7b57d44f-9p9pc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6886b96f9 from 7 to 6 42m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 42m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-97z9h 41m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-97z9h Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-6f8856-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-6f8856-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Killing pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-fd1dfc-openshift-default-75bbcc65dc-jcvjv Stopping container istio-proxy 41m Normal AddedInterface pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Add eth0 [10.133.0.180/23] from ovn-kubernetes 41m Normal AddedLabel gateway/e2e-shared-a-w1-476a67 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-476a67 41m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7 Created pod: e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz 41m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-476a67-openshift-default Scaled up replica set e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7 from 0 to 1 41m Warning Unhealthy pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-inf-b-c6d97d-openshift-default-79fc67f7c9-k5r4d Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-476a67: serviceaccount "payload-processing-e2e-shared-a-w1-476a67" not found 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c Error creating: pods "payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-476a67: serviceaccount "payload-processing-e2e-shared-a-w1-476a67" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-476a67 Scaled up replica set payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9 Created pod: payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-476a67 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c Created pod: payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl Add eth0 [10.133.0.181/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj Add eth0 [10.133.0.182/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj Container created 41m Normal Started pod/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj Container started 41m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl Container created 41m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl Container started 41m Normal Killing pod/payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t Stopping container payload-pre-processing 41m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-f2bdf3-openshift-default Scaled down replica set e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 from 2 to 1 41m Normal Killing pod/payload-processing-e2e-inf-b-609843-b8d75c975-frvqn Stopping container payload-processing 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-f2bdf3-openshift-default New size: 1; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767 Deleted pod: e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf 41m Normal Killing pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f Deleted pod: e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-b024b5-openshift-default New size: 2; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj 41m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 3 to 2 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-b024b5-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-b024b5-openshift-default-75d4b8c46f from 3 to 2 41m Warning ProbeError pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-tv6bf Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/payload-processing-e2e-inf-b-609843-b8d75c975-frvqn Readiness probe failed: timeout: failed to connect service "10.133.0.142:50052" within 1s: context deadline exceeded 41m Warning ProbeError pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-2x457 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-4fsjj Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-s7c69 41m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 41m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-s7c69 Stopping container discovery 41m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 41m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-609843-79599884c-vjw2t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal Killing pod/e2e-inf-b-609843-openshift-default-db7868f45-jmbpz Stopping container istio-proxy 40m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99 Created pod: e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt 40m Normal AddedLabel gateway/e2e-shared-a-w5-99a5c2 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-99a5c2 40m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-99a5c2-openshift-default Scaled up replica set e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99 from 0 to 1 40m Normal AddedInterface pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Add eth0 [10.134.0.185/23] from ovn-kubernetes 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f Error creating: pods "payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-99a5c2: serviceaccount "payload-processing-e2e-shared-a-w5-99a5c2" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-99a5c2 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c Created pod: payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-nvpzl 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-99a5c2 Scaled up replica set payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c Error creating: pods "payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-99a5c2: serviceaccount "payload-processing-e2e-shared-a-w5-99a5c2" not found 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-nvpzl Add eth0 [10.133.0.184/23] from ovn-kubernetes 40m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-nvpzl Container created 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f Created pod: payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr 40m Normal Started pod/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr Container started 40m Normal Created pod/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr Container created 40m Normal Pulled pod/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-nvpzl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr Add eth0 [10.133.0.185/23] from ovn-kubernetes 40m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-nvpzl Container started 40m Warning Unhealthy pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Readiness probe failed: Get "http://10.133.0.180:15021/healthz/ready": read tcp 10.133.0.2:34486->10.133.0.180:15021: read: connection reset by peer 40m Warning ProbeError pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Readiness probe error: Get "http://10.133.0.180:15021/healthz/ready": read tcp 10.133.0.2:34486->10.133.0.180:15021: read: connection reset by peer... 40m Normal Pulled pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Container created 40m Normal Started pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Container started 40m Normal AddedLabel gateway/e2e-shared-b-w1-b8fd5a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-b8fd5a 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-b8fd5a-openshift-default Scaled up replica set e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7 from 0 to 1 40m Normal AddedInterface pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Add eth0 [10.132.0.198/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7 Created pod: e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w 40m Warning Unhealthy pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Startup probe failed: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Startup probe error: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused... 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547 Created pod: payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c Created pod: payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-b8fd5a: serviceaccount "payload-processing-e2e-shared-b-w1-b8fd5a" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-b8fd5a Scaled up replica set payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-b8fd5a Scaled up replica set payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c from 0 to 1 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t Add eth0 [10.133.0.186/23] from ovn-kubernetes 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c Error creating: pods "payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-b8fd5a: serviceaccount "payload-processing-e2e-shared-b-w1-b8fd5a" not found 39m Normal Created pod/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 Container created 39m Normal Started pod/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 Container started 39m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 Add eth0 [10.133.0.187/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t Container started 39m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t Container created 39m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Killing pod/payload-processing-e2e-inf-a-762753-64f6f4597c-vnz4m Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs Stopping container payload-pre-processing 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-762753-6847dc7c9f-xxdcs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal Killing pod/e2e-inf-a-762753-openshift-default-85767dbb67-6fxtg Stopping container istio-proxy 39m Normal Killing pod/e2e-inf-a-762753-openshift-default-85767dbb67-8dghm Stopping container istio-proxy 39m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99 Created pod: e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-99a5c2-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-99a5c2-openshift-default Scaled up replica set e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99 from 1 to 2 39m Normal Pulled pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Killing pod/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj Stopping container payload-processing 39m Normal Created pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Container created 39m Normal Created pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Container created 39m Normal Started pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Container started 39m Normal Pulled pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Add eth0 [10.132.0.202/23] from ovn-kubernetes 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm Stopping container payload-pre-processing 39m Normal Started pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Container started 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9e377-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-c9e377-openshift-default-bbb7b4499 from 2 to 1 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9e377-openshift-default-bbb7b4499 Deleted pod: e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-476a67-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9e377-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-476a67-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Stopping container istio-proxy 39m Warning Unhealthy pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Startup probe failed: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Startup probe error: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused... 39m Normal AddedLabel gateway/e2e-shared-b-w5-328d61 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-328d61 39m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-328d61-openshift-default Scaled up replica set e2e-shared-b-w5-328d61-openshift-default-7857fc774c from 0 to 1 39m Warning Unhealthy pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Startup probe failed: Get "http://10.134.0.185:15021/healthz/ready": dial tcp 10.134.0.185:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Startup probe error: Get "http://10.134.0.185:15021/healthz/ready": dial tcp 10.134.0.185:15021: connect: connection refused... 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-328d61-openshift-default-7857fc774c Created pod: e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w 39m Normal AddedInterface pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Add eth0 [10.132.0.204/23] from ovn-kubernetes 39m Warning Unhealthy pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-d7cbd Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Killing pod/payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 Stopping container payload-processing 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-328d61 Scaled up replica set payload-processing-e2e-shared-b-w5-328d61-74b757865c from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-328d61 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f Created pod: payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-328d61-74b757865c Created pod: payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw 39m Normal Killing pod/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b Stopping container payload-pre-processing 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f Error creating: pods "payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-328d61: serviceaccount "payload-processing-e2e-shared-b-w5-328d61" not found 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-328d61-74b757865c Error creating: pods "payload-processing-e2e-shared-b-w5-328d61-74b757865c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-328d61: serviceaccount "payload-processing-e2e-shared-b-w5-328d61" not found 39m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc Container created 39m Normal Started pod/payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw Container started 39m Normal Created pod/payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw Container created 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc Add eth0 [10.133.0.189/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc Container started 39m Normal Pulled pod/payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw Add eth0 [10.133.0.188/23] from ovn-kubernetes 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-9de7f5-85bdf664d7-kxrlm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/payload-processing-e2e-inf-b-9de7f5-6b8cc55f75-txjxj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-99a5c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-99a5c2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-b87c62-7745bdc775-cd27b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/payload-processing-e2e-inf-a-b87c62-6b8656d99c-ph244 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal Killing pod/e2e-inf-a-b87c62-openshift-default-79f485ccf5-qnlfk Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Stopping container istio-proxy 38m Normal Killing pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-mwqkt Stopping container istio-proxy 38m Normal Killing pod/payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 Stopping container payload-pre-processing 38m Warning ProbeError pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-inf-b-9de7f5-openshift-default-84c9b4bf8f-l5qkv Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal ScalingReplicaSet deployment/e2e-worker-w2-735123-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-735123-openshift-default-5889966f5 from 2 to 1 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-735123-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-735123-openshift-default-5889966f5 Deleted pod: e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g 38m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w6-be301b-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-be301b-openshift-default-7bd4946d5f from 2 to 1 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-671959-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w0-671959-openshift-default-586d4c9cc5 Deleted pod: e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-be301b-openshift-default-7bd4946d5f Deleted pod: e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be301b-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w0-671959-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-671959-openshift-default-586d4c9cc5 from 2 to 1 38m Warning ProbeError pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-worker-w2-735123-openshift-default-5889966f5-xnm5g Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning Unhealthy pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-qxbwd Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning ProbeError pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-zpj5m Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-01a117-96d7f8b99-4c9m8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/payload-processing-e2e-inf-b-01a117-849b4fc645-28vbf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal Killing pod/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 Stopping container payload-pre-processing 37m Normal Killing pod/e2e-inf-b-01a117-openshift-default-56c5c46dc5-r9jtz Stopping container istio-proxy 37m Warning Unhealthy pod/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal Killing pod/e2e-inf-a-72120c-openshift-default-77d78ddc45-gh6gg Stopping container istio-proxy 37m Normal Pulled pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Readiness probe failed: Get "http://10.132.0.198:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Normal Started pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Container started 37m Normal Created pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Container created 37m Warning ProbeError pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Readiness probe error: Get "http://10.132.0.198:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Normal Created pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Container created 37m Normal Started pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Container started 37m Normal Pulled pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-72120c-7776cf45f-xlr79 Readiness probe failed: timeout: failed to connect service "10.133.0.156:50052" within 1s: context deadline exceeded 37m Warning Unhealthy pod/payload-processing-e2e-inf-a-72120c-5d4968bd99-fz4dd Readiness probe failed: timeout: failed to connect service "10.133.0.157:50052" within 1s: context deadline exceeded 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-328d61-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-328d61-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning ProbeError pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Startup probe error: Get "http://10.132.0.204:15021/healthz/ready": dial tcp 10.132.0.204:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Startup probe failed: Get "http://10.132.0.204:15021/healthz/ready": dial tcp 10.132.0.204:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Startup probe failed: Get "http://10.132.0.198:15021/healthz/ready": dial tcp 10.132.0.198:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Startup probe error: Get "http://10.132.0.198:15021/healthz/ready": dial tcp 10.132.0.198:15021: connect: connection refused... 37m Normal Killing pod/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-inf-b-82b5e8-5cc6c765cf-4z7ng Stopping container payload-processing 37m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6886b96f9 from 6 to 4 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-c7jth 37m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Stopping container istio-proxy 37m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-vlrk2 37m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 37m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-c7jth Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-vlrk2 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0d8bae-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-0d8bae-openshift-default-6c96b68449 Deleted pod: e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt 37m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-0d8bae-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-0d8bae-openshift-default-6c96b68449 from 2 to 1 37m Warning ProbeError pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-dq5tt Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-82b5e8-77c7fb89bc-2cvn8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-b8fd5a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-b8fd5a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-328d61-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-328d61-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-gmlkt Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-b-82b5e8-openshift-default-79b8ff867c-4wp65 Stopping container istio-proxy 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-d10721-openshift-default-bf897765f Created pod: e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s 36m Normal AddedLabel gateway/e2e-shared-a-w3-d10721 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-d10721 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-d10721-openshift-default Scaled up replica set e2e-shared-a-w3-d10721-openshift-default-bf897765f from 0 to 1 36m Normal AddedInterface pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Add eth0 [10.132.0.217/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-d10721 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-d10721 Scaled up replica set payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc from 0 to 1 36m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x Container created 36m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59 Created pod: payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x 36m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x Container started 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-d10721: serviceaccount "payload-processing-e2e-shared-a-w3-d10721" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc Created pod: payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc Error creating: pods "payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-d10721: serviceaccount "payload-processing-e2e-shared-a-w3-d10721" not found 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x Add eth0 [10.133.0.193/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl Add eth0 [10.133.0.194/23] from ovn-kubernetes 36m Normal Created pod/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl Container created 36m Normal Started pod/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl Container started 36m Normal Started pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 Container started 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-498580-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal Pulled pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-498580-openshift-default Scaled up replica set e2e-shared-a-w0-498580-openshift-default-b66db7999 from 1 to 2 36m Normal Created pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 Container created 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-498580-openshift-default-b66db7999 Created pod: e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 36m Normal AddedInterface pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 Add eth0 [10.132.0.219/23] from ovn-kubernetes 36m Normal Killing pod/payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 Stopping container payload-pre-processing 36m Warning Unhealthy pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused... 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7 Created pod: e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-b8fd5a-openshift-default Scaled up replica set e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7 from 1 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-b8fd5a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal Pulled pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 Container started 36m Normal Created pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 Container created 36m Normal AddedInterface pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 Add eth0 [10.133.0.195/23] from ovn-kubernetes 36m Warning Unhealthy pod/payload-processing-e2e-inf-a-57e594-86cdf6574c-dt5v2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-57e594-6784f95c8c-cw4m2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal Killing pod/e2e-inf-a-57e594-openshift-default-55cf466c9c-c8tb9 Stopping container istio-proxy 36m Warning ProbeError pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused 36m Normal Killing pod/payload-processing-e2e-inf-b-4f4e69-5f8fb86855-mzzfn Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 Stopping container payload-pre-processing 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-4f4e69-579b866c9-5rhl2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c Created pod: e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b 35m Normal AddedLabel gateway/e2e-shared-b-w3-ada0c1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-ada0c1 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-ada0c1-openshift-default Scaled up replica set e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c from 0 to 1 35m Normal AddedInterface pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b Add eth0 [10.132.0.226/23] from ovn-kubernetes 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k Stopping container payload-processing 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-ada0c1 Scaled up replica set payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-ada0c1 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5 from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85 Error creating: pods "payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-ada0c1: serviceaccount "payload-processing-e2e-shared-b-w3-ada0c1" not found 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-ada0c1: serviceaccount "payload-processing-e2e-shared-b-w3-ada0c1" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85 Created pod: payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc 35m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc Add eth0 [10.133.0.196/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb Add eth0 [10.133.0.197/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb Container created 35m Normal Started pod/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc Container started 35m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb Container started 35m Normal Created pod/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc Container created 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5 Created pod: payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb 35m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Killing pod/e2e-inf-b-4f4e69-openshift-default-58bcbfbb57-7q48t Stopping container istio-proxy 35m Normal Started pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Container started 35m Normal Created pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Container created 35m Warning ProbeError pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Readiness probe error: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Readiness probe failed: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused 35m Normal Pulled pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-ada0c1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-ada0c1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning ProbeError pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Startup probe error: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Startup probe failed: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused 35m Normal Killing pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-inf-a-c942c9-openshift-default-5bdd576867-mjfz2 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c942c9-6f77df795f-5mbwk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/payload-processing-e2e-inf-a-c942c9-68f5b49c89-pfr2k Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-d10721-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-d10721-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Created pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds Container created 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-328d61-openshift-default Scaled up replica set e2e-shared-b-w5-328d61-openshift-default-7857fc774c from 1 to 2 35m Normal Started pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds Container started 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-328d61-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal Pulled pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-328d61-openshift-default-7857fc774c Created pod: e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds 35m Normal AddedInterface pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds Add eth0 [10.134.0.189/23] from ovn-kubernetes 35m Normal Killing pod/payload-processing-e2e-inf-b-6f8856-647b85df9c-tbspl Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt Stopping container payload-pre-processing 35m Normal ScalingReplicaSet deployment/e2e-inf-b-6f8856-openshift-default Scaled up replica set e2e-inf-b-6f8856-openshift-default-6f9ff5d847 from 1 to 2 35m Normal Created pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Container created 35m Normal Pulled pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Container started 35m Normal SuccessfulCreate replicaset/e2e-inf-b-6f8856-openshift-default-6f9ff5d847 Created pod: e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 35m Normal AddedInterface pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Add eth0 [10.133.0.198/23] from ovn-kubernetes 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-6f8856-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Warning ProbeError pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds Startup probe error: Get "http://10.134.0.189:15021/healthz/ready": dial tcp 10.134.0.189:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds Startup probe failed: Get "http://10.134.0.189:15021/healthz/ready": dial tcp 10.134.0.189:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Startup probe failed: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Startup probe error: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused... 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-d10721-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-d10721-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-6f8856-9c858b495-pgqmt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Stopping container istio-proxy 34m Warning ProbeError pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal AddedLabel gateway/e2e-ipp-a-73f915 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-73f915 34m Warning Unhealthy pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-kkklt Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal ScalingReplicaSet deployment/e2e-ipp-a-73f915-openshift-default Scaled up replica set e2e-ipp-a-73f915-openshift-default-64b6f4c569 from 0 to 1 34m Normal AddedInterface pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Add eth0 [10.133.0.200/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/e2e-ipp-a-73f915-openshift-default-64b6f4c569 Created pod: e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j 34m Warning ProbeError pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-inf-b-6f8856-openshift-default-6f9ff5d847-rr9z4 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-73f915-78fccdbd7f Error creating: pods "payload-processing-e2e-ipp-a-73f915-78fccdbd7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73f915: serviceaccount "payload-processing-e2e-ipp-a-73f915" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-73f915 Scaled up replica set payload-processing-e2e-ipp-a-73f915-78fccdbd7f from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-73f915-d658955ff Error creating: pods "payload-pre-processing-e2e-ipp-a-73f915-d658955ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73f915: serviceaccount "payload-processing-e2e-ipp-a-73f915" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-73f915-d658955ff Created pod: payload-pre-processing-e2e-ipp-a-73f915-d658955ff-lphgt 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-73f915 Scaled up replica set payload-pre-processing-e2e-ipp-a-73f915-d658955ff from 0 to 1 34m Normal Created pod/payload-pre-processing-e2e-ipp-a-73f915-d658955ff-lphgt Container created 34m Normal Pulled pod/payload-processing-e2e-ipp-a-73f915-78fccdbd7f-98rsg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-ipp-a-73f915-78fccdbd7f-98rsg Add eth0 [10.134.0.191/23] from ovn-kubernetes 34m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-73f915-d658955ff-lphgt Add eth0 [10.133.0.201/23] from ovn-kubernetes 34m Normal Started pod/payload-processing-e2e-ipp-a-73f915-78fccdbd7f-98rsg Container started 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-73f915-78fccdbd7f Created pod: payload-processing-e2e-ipp-a-73f915-78fccdbd7f-98rsg 34m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-73f915-d658955ff-lphgt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-pre-processing-e2e-ipp-a-73f915-d658955ff-lphgt Container started 34m Normal Created pod/payload-processing-e2e-ipp-a-73f915-78fccdbd7f-98rsg Container created 33m Normal Created pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b Container created 33m Normal Pulled pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b Container started 33m Warning ProbeError pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b Startup probe error: Get "http://10.132.0.226:15021/healthz/ready": dial tcp 10.132.0.226:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b Startup probe failed: Get "http://10.132.0.226:15021/healthz/ready": dial tcp 10.132.0.226:15021: connect: connection refused 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-ada0c1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-ada0c1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/payload-processing-e2e-ipp-a-73f915-78fccdbd7f-98rsg Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-ipp-a-73f915-d658955ff-lphgt Stopping container payload-pre-processing 33m Warning Unhealthy pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Readiness probe failed: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Readiness probe error: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Normal Pulled pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Container created 33m Normal Started pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Container started 33m Warning Unhealthy pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused 33m Warning ProbeError pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused... 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-73f915-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-73f915-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-73f915-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-73f915-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-73f915-6f7644dc56 Error creating: pods "payload-processing-e2e-ipp-a-73f915-6f7644dc56-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73f915: serviceaccount "payload-processing-e2e-ipp-a-73f915" not found 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-73f915-555585ff79 Error creating: pods "payload-pre-processing-e2e-ipp-a-73f915-555585ff79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73f915: serviceaccount "payload-processing-e2e-ipp-a-73f915" not found 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-73f915 Scaled up replica set payload-processing-e2e-ipp-a-73f915-6f7644dc56 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-73f915 Scaled up replica set payload-pre-processing-e2e-ipp-a-73f915-555585ff79 from 0 to 1 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st Add eth0 [10.134.0.193/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-73f915-6f7644dc56 Created pod: payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx 32m Normal Pulling pod/payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 32m Normal AddedInterface pod/payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx Add eth0 [10.132.0.229/23] from ovn-kubernetes 32m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-73f915-555585ff79 Created pod: payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st 32m Normal Started pod/payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx Container started 32m Normal Started pod/payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st Container started 32m Normal Created pod/payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st Container created 32m Normal Pulled pod/payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.346s (1.346s including waiting). Image size: 166620835 bytes. 32m Normal Created pod/payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx Container created 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.655s (1.655s including waiting). Image size: 166620835 bytes. 32m Normal AddedLabel gateway/e2e-ipp-b-75ead8 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-75ead8 32m Normal ScalingReplicaSet deployment/e2e-ipp-b-75ead8-openshift-default Scaled up replica set e2e-ipp-b-75ead8-openshift-default-65576bc997 from 0 to 1 32m Normal AddedInterface pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Add eth0 [10.134.0.194/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/e2e-ipp-b-75ead8-openshift-default-65576bc997 Created pod: e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-75ead8-5b87c69f95 Error creating: pods "payload-processing-e2e-ipp-b-75ead8-5b87c69f95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-75ead8: serviceaccount "payload-processing-e2e-ipp-b-75ead8" not found 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5 Error creating: pods "payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-75ead8: serviceaccount "payload-processing-e2e-ipp-b-75ead8" not found 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-75ead8 Scaled up replica set payload-processing-e2e-ipp-b-75ead8-5b87c69f95 from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5 Created pod: payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-qtr54 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-75ead8-5b87c69f95 Created pod: payload-processing-e2e-ipp-b-75ead8-5b87c69f95-vt2vm 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-75ead8 Scaled up replica set payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5 from 0 to 1 32m Normal Started pod/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-qtr54 Container started 32m Normal Created pod/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-qtr54 Container created 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-qtr54 Add eth0 [10.133.0.204/23] from ovn-kubernetes 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-qtr54 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-processing-e2e-ipp-b-75ead8-5b87c69f95-vt2vm Container created 32m Normal AddedInterface pod/payload-processing-e2e-ipp-b-75ead8-5b87c69f95-vt2vm Add eth0 [10.133.0.205/23] from ovn-kubernetes 32m Normal Pulled pod/payload-processing-e2e-ipp-b-75ead8-5b87c69f95-vt2vm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-processing-e2e-ipp-b-75ead8-5b87c69f95-vt2vm Container started 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-75ead8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-75ead8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal SuccessfulCreate replicaset/e2e-ipp-a-e41918-openshift-default-5ccdddb467 Created pod: e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq 31m Normal ScalingReplicaSet deployment/e2e-ipp-a-e41918-openshift-default Scaled up replica set e2e-ipp-a-e41918-openshift-default-5ccdddb467 from 0 to 1 31m Normal AddedLabel gateway/e2e-ipp-a-e41918 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-e41918 31m Normal AddedInterface pod/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq Add eth0 [10.132.0.238/23] from ovn-kubernetes 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5 Error creating: pods "payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e41918: serviceaccount "payload-processing-e2e-ipp-a-e41918" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e41918-7f99fb8c89 Created pod: payload-processing-e2e-ipp-a-e41918-7f99fb8c89-7lq8d 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e41918 Scaled up replica set payload-pre-processing-e2e-ipp-a-e41918-6686c749f5 from 0 to 1 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e41918-7f99fb8c89 Error creating: pods "payload-processing-e2e-ipp-a-e41918-7f99fb8c89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e41918: serviceaccount "payload-processing-e2e-ipp-a-e41918" not found 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5 Created pod: payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-cbh5z 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e41918 Scaled up replica set payload-processing-e2e-ipp-a-e41918-7f99fb8c89 from 0 to 1 31m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e41918-7f99fb8c89-7lq8d Add eth0 [10.133.0.207/23] from ovn-kubernetes 31m Normal Killing pod/payload-processing-e2e-ipp-b-75ead8-5b87c69f95-vt2vm Stopping container payload-processing 31m Normal Started pod/payload-processing-e2e-ipp-a-e41918-7f99fb8c89-7lq8d Container started 31m Normal Created pod/payload-processing-e2e-ipp-a-e41918-7f99fb8c89-7lq8d Container created 31m Normal Killing pod/payload-pre-processing-e2e-ipp-b-75ead8-f9ccf6c5-qtr54 Stopping container payload-pre-processing 31m Normal Pulled pod/payload-processing-e2e-ipp-a-e41918-7f99fb8c89-7lq8d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Started pod/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-cbh5z Container started 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-cbh5z Add eth0 [10.133.0.206/23] from ovn-kubernetes 31m Normal Created pod/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-cbh5z Container created 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-cbh5z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-75ead8 Scaled up replica set payload-processing-e2e-ipp-b-75ead8-7c5767987b from 0 to 1 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-75ead8 Scaled up replica set payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67 from 0 to 1 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-75ead8-7c5767987b Error creating: pods "payload-processing-e2e-ipp-b-75ead8-7c5767987b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-75ead8: serviceaccount "payload-processing-e2e-ipp-b-75ead8" not found 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc Add eth0 [10.133.0.208/23] from ovn-kubernetes 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67 Error creating: pods "payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-75ead8: serviceaccount "payload-processing-e2e-ipp-b-75ead8" not found 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-75ead8-7c5767987b Created pod: payload-processing-e2e-ipp-b-75ead8-7c5767987b-gp4gv 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67 Created pod: payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc 30m Normal Pulled pod/payload-processing-e2e-ipp-b-75ead8-7c5767987b-gp4gv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-ipp-b-75ead8-7c5767987b-gp4gv Add eth0 [10.132.0.241/23] from ovn-kubernetes 30m Normal Created pod/payload-processing-e2e-ipp-b-75ead8-7c5767987b-gp4gv Container created 30m Normal Pulling pod/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 30m Normal Started pod/payload-processing-e2e-ipp-b-75ead8-7c5767987b-gp4gv Container started 30m Normal Created pod/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc Container created 30m Normal Started pod/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc Container started 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.722s (1.722s including waiting). Image size: 166620835 bytes. 30m Warning ProbeError pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Readiness probe error: Get "http://10.134.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 30m Normal ScalingReplicaSet deployment/e2e-ipp-a-4207cd-openshift-default Scaled up replica set e2e-ipp-a-4207cd-openshift-default-58cb5b4b55 from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55 Created pod: e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn 30m Normal AddedLabel gateway/e2e-ipp-a-4207cd Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-4207cd 30m Warning Unhealthy pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Readiness probe failed: Get "http://10.134.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 30m Normal AddedInterface pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Add eth0 [10.132.0.242/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Container created 30m Normal Started pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Container started 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-714f8f-openshift-default Scaled up replica set e2e-shared-b-w6-714f8f-openshift-default-85496857c7 from 1 to 2 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-714f8f-openshift-default-85496857c7 Created pod: e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c 30m Normal AddedInterface pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c Add eth0 [10.134.0.196/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6886b96f9 from 4 to 3 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-714f8f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 30m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-62hfr 30m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Stopping container istio-proxy 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-4207cd Scaled up replica set payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7 from 0 to 1 30m Normal Started pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c Container started 30m Normal Created pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c Container created 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-4207cd Scaled up replica set payload-processing-e2e-ipp-a-4207cd-f974fcd45 from 0 to 1 30m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7 Error creating: pods "payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4207cd: serviceaccount "payload-processing-e2e-ipp-a-4207cd" not found 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-4207cd-f974fcd45 Created pod: payload-processing-e2e-ipp-a-4207cd-f974fcd45-22f7w 30m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-62hfr Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-nr6dc Add eth0 [10.133.0.210/23] from ovn-kubernetes 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-nr6dc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-4207cd-f974fcd45 Error creating: pods "payload-processing-e2e-ipp-a-4207cd-f974fcd45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4207cd: serviceaccount "payload-processing-e2e-ipp-a-4207cd" not found 30m Normal Pulled pod/payload-processing-e2e-ipp-a-4207cd-f974fcd45-22f7w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-nr6dc Container created 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-nr6dc Container started 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7 Created pod: payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-nr6dc 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-4207cd-f974fcd45-22f7w Add eth0 [10.133.0.211/23] from ovn-kubernetes 30m Normal AddedInterface pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl Add eth0 [10.134.0.197/23] from ovn-kubernetes 30m Normal Created pod/payload-processing-e2e-ipp-a-4207cd-f974fcd45-22f7w Container created 30m Normal Started pod/payload-processing-e2e-ipp-a-4207cd-f974fcd45-22f7w Container started 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-498580-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-498580-openshift-default-b66db7999 Created pod: e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-498580-openshift-default Scaled up replica set e2e-shared-a-w0-498580-openshift-default-b66db7999 from 2 to 3 30m Normal Created pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl Container created 30m Normal Started pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl Container started 30m Normal Pulled pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Warning ProbeError pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c Startup probe error: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c Startup probe failed: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl Startup probe failed: Get "http://10.134.0.197:15021/healthz/ready": dial tcp 10.134.0.197:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl Startup probe error: Get "http://10.134.0.197:15021/healthz/ready": dial tcp 10.134.0.197:15021: connect: connection refused... 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-36af31-openshift-default Scaled up replica set e2e-shared-a-w2-36af31-openshift-default-74f58f595f from 1 to 2 30m Normal Started pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Container started 30m Normal AddedInterface pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Add eth0 [10.132.0.245/23] from ovn-kubernetes 30m Normal Created pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Container created 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-36af31-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal Pulled pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-36af31-openshift-default-74f58f595f Created pod: e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx 29m Warning ProbeError pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Startup probe error: Get "http://10.132.0.245:15021/healthz/ready": dial tcp 10.132.0.245:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Startup probe failed: Get "http://10.132.0.245:15021/healthz/ready": dial tcp 10.132.0.245:15021: connect: connection refused 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-daaf26-openshift-default New size: 1; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w3-daaf26-openshift-default Scaled down replica set e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 from 2 to 1 29m Normal SuccessfulDelete replicaset/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55 Deleted pod: e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx 29m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-pq9nx Stopping container istio-proxy 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-75ead8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-75ead8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-bcdb4e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-bcdb4e-openshift-default Scaled up replica set e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687 from 1 to 2 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687 Created pod: e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s 29m Normal Pulled pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s Container started 29m Normal Created pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s Container created 29m Normal AddedInterface pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s Add eth0 [10.133.0.212/23] from ovn-kubernetes 29m Warning Unhealthy pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s Startup probe failed: Get "http://10.133.0.212:15021/healthz/ready": dial tcp 10.133.0.212:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s Startup probe error: Get "http://10.133.0.212:15021/healthz/ready": dial tcp 10.133.0.212:15021: connect: connection refused... 29m Normal Killing pod/payload-processing-e2e-ipp-a-e41918-7f99fb8c89-7lq8d Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e41918-6686c749f5-cbh5z Stopping container payload-pre-processing 29m Normal SuccessfulCreate replicaset/e2e-ipp-b-33d44f-openshift-default-55d7454777 Created pod: e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 29m Normal ScalingReplicaSet deployment/e2e-ipp-b-33d44f-openshift-default Scaled up replica set e2e-ipp-b-33d44f-openshift-default-55d7454777 from 0 to 1 29m Normal AddedLabel gateway/e2e-ipp-b-33d44f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-33d44f 29m Normal AddedInterface pod/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 Add eth0 [10.133.0.213/23] from ovn-kubernetes 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f Error creating: pods "payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-33d44f: serviceaccount "payload-processing-e2e-ipp-b-33d44f" not found 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-33d44f-b69895595 Error creating: pods "payload-processing-e2e-ipp-b-33d44f-b69895595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-33d44f: serviceaccount "payload-processing-e2e-ipp-b-33d44f" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f Created pod: payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-jnr6j 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-33d44f Scaled up replica set payload-processing-e2e-ipp-b-33d44f-b69895595 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-33d44f Scaled up replica set payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-33d44f-b69895595 Created pod: payload-processing-e2e-ipp-b-33d44f-b69895595-65qmw 29m Normal AddedInterface pod/payload-processing-e2e-ipp-b-33d44f-b69895595-65qmw Add eth0 [10.133.0.216/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-jnr6j Add eth0 [10.133.0.214/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-ipp-b-33d44f-b69895595-65qmw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-processing-e2e-ipp-b-33d44f-b69895595-65qmw Container created 29m Normal Started pod/payload-processing-e2e-ipp-b-33d44f-b69895595-65qmw Container started 29m Normal Started pod/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-jnr6j Container started 29m Normal Created pod/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-jnr6j Container created 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-jnr6j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-73f915-555585ff79-9k9st Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-ipp-a-73f915-6f7644dc56-wchnx Stopping container payload-processing 29m Normal Pulled pod/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq Container started 29m Normal Created pod/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq Container created 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e41918-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e41918-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning ProbeError pod/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq Startup probe error: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq Startup probe failed: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused 29m Normal Killing pod/e2e-ipp-a-73f915-openshift-default-64b6f4c569-qq92j Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e41918 Scaled up replica set payload-processing-e2e-ipp-a-e41918-766c7f5686 from 0 to 1 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc Error creating: pods "payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e41918: serviceaccount "payload-processing-e2e-ipp-a-e41918" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e41918-766c7f5686 Created pod: payload-processing-e2e-ipp-a-e41918-766c7f5686-8mf57 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc Created pod: payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-2ln6h 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e41918 Scaled up replica set payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc from 0 to 1 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e41918-766c7f5686 Error creating: pods "payload-processing-e2e-ipp-a-e41918-766c7f5686-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e41918: serviceaccount "payload-processing-e2e-ipp-a-e41918" not found 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-2ln6h Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99 Created pod: e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-3c90d8-openshift-default Scaled up replica set e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99 from 1 to 2 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e41918-766c7f5686-8mf57 Add eth0 [10.133.0.218/23] from ovn-kubernetes 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-3c90d8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-2ln6h Container started 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-2ln6h Add eth0 [10.133.0.217/23] from ovn-kubernetes 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-2ln6h Container created 29m Normal Created pod/payload-processing-e2e-ipp-a-e41918-766c7f5686-8mf57 Container created 29m Normal Started pod/payload-processing-e2e-ipp-a-e41918-766c7f5686-8mf57 Container started 29m Normal Pulled pod/payload-processing-e2e-ipp-a-e41918-766c7f5686-8mf57 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 29m Normal SuccessfulCreate replicaset/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7 Created pod: e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 29m Normal AddedInterface pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt Add eth0 [10.134.0.198/23] from ovn-kubernetes 29m Normal AddedLabel gateway/e2e-ipp-a-fa97ea Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-fa97ea 29m Normal ScalingReplicaSet deployment/e2e-ipp-a-fa97ea-openshift-default Scaled up replica set e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7 from 0 to 1 29m Normal Pulled pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Add eth0 [10.132.0.250/23] from ovn-kubernetes 29m Normal Created pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt Container created 29m Normal Started pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt Container started 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655 Error creating: pods "payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-fa97ea: serviceaccount "payload-processing-e2e-ipp-a-fa97ea" not found 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-fa97ea Scaled up replica set payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-fa97ea Scaled up replica set payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9 from 0 to 1 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9 Error creating: pods "payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-fa97ea: serviceaccount "payload-processing-e2e-ipp-a-fa97ea" not found 28m Warning ProbeError pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt Startup probe error: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt Startup probe failed: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-qlmj4 Add eth0 [10.132.0.251/23] from ovn-kubernetes 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-qlmj4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9 Created pod: payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-mt8xz 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-qlmj4 Container started 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-qlmj4 Container created 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655 Created pod: payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-qlmj4 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-mt8xz Add eth0 [10.134.0.199/23] from ovn-kubernetes 28m Normal Created pod/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-mt8xz Container created 28m Normal Started pod/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-mt8xz Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-mt8xz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Pulled pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Container created 28m Normal Started pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Container started 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e41918-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e41918-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning ProbeError pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Startup probe error: Get "http://10.132.0.242:15021/healthz/ready": dial tcp 10.132.0.242:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Startup probe failed: Get "http://10.132.0.242:15021/healthz/ready": dial tcp 10.132.0.242:15021: connect: connection refused 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-4207cd-6dcf849dd7-nr6dc Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-ipp-a-4207cd-f974fcd45-22f7w Stopping container payload-processing 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-4207cd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-4207cd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Killing pod/payload-processing-e2e-ipp-b-75ead8-7c5767987b-gp4gv Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-b-75ead8-7c6d545b67-hggkc Stopping container payload-pre-processing 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-4207cd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-4207cd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Killing pod/e2e-ipp-b-75ead8-openshift-default-65576bc997-jm6sm Stopping container istio-proxy 28m Normal Killing pod/payload-processing-e2e-ipp-b-33d44f-b69895595-65qmw Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-b-33d44f-6485f5f65f-jnr6j Stopping container payload-pre-processing 28m Normal Killing pod/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 Stopping container payload-processing 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc Error creating: pods "payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4207cd: serviceaccount "payload-processing-e2e-ipp-a-4207cd" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-4207cd Scaled up replica set payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-4207cd Scaled up replica set payload-processing-e2e-ipp-a-4207cd-55648f6b98 from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-4207cd-55648f6b98 Error creating: pods "payload-processing-e2e-ipp-a-4207cd-55648f6b98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4207cd: serviceaccount "payload-processing-e2e-ipp-a-4207cd" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc Created pod: payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-ndr4b 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-4207cd-55648f6b98-x88sq Add eth0 [10.133.0.225/23] from ovn-kubernetes 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-ndr4b Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-ndr4b Add eth0 [10.133.0.224/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-4207cd-55648f6b98 Created pod: payload-processing-e2e-ipp-a-4207cd-55648f6b98-x88sq 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-ndr4b Container started 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-ndr4b Container created 28m Normal Pulled pod/payload-processing-e2e-ipp-a-4207cd-55648f6b98-x88sq Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-processing-e2e-ipp-a-4207cd-55648f6b98-x88sq Container created 28m Normal Started pod/payload-processing-e2e-ipp-a-4207cd-55648f6b98-x88sq Container started 27m Normal Started pod/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 Container started 27m Normal Pulled pod/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 Container created 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-33d44f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-33d44f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning Unhealthy pod/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 Startup probe failed: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused 27m Warning ProbeError pod/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 Startup probe error: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused... 27m Normal AddedLabel gateway/e2e-ipp-b-d68696 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-d68696 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-d68696-openshift-default Scaled up replica set e2e-ipp-b-d68696-openshift-default-6dd844fcf from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-d68696-openshift-default-6dd844fcf Created pod: e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf 27m Normal AddedInterface pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Add eth0 [10.132.1.12/23] from ovn-kubernetes 27m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-0d8bae-85c849467-dws9w Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-worker-w4-0d8bae-5b464cf9d7-vlsb9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-d68696 Scaled up replica set payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7 from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-d68696 Scaled up replica set payload-processing-e2e-ipp-b-d68696-884f5657 from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-d68696-884f5657 Error creating: pods "payload-processing-e2e-ipp-b-d68696-884f5657-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d68696: serviceaccount "payload-processing-e2e-ipp-b-d68696" not found 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7 Error creating: pods "payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d68696: serviceaccount "payload-processing-e2e-ipp-b-d68696" not found 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-d68696-884f5657 Created pod: payload-processing-e2e-ipp-b-d68696-884f5657-t256p 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7 Created pod: payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-bwqn2 27m Normal Created pod/payload-processing-e2e-ipp-b-d68696-884f5657-t256p Container created 27m Normal Started pod/payload-processing-e2e-ipp-b-d68696-884f5657-t256p Container started 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-bwqn2 Add eth0 [10.132.1.13/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-bwqn2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-bwqn2 Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-bwqn2 Container started 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-d68696-884f5657-t256p Add eth0 [10.133.0.226/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-b-d68696-884f5657-t256p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Killing pod/payload-processing-e2e-ipp-a-fa97ea-5bb454dbc9-mt8xz Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-ipp-a-fa97ea-58c4c47655-qlmj4 Stopping container payload-pre-processing 27m Warning ProbeError pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Readiness probe error: Get "http://10.132.0.250:15021/healthz/ready": read tcp 10.132.0.2:56434->10.132.0.250:15021: read: connection reset by peer... 27m Warning Unhealthy pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Readiness probe failed: Get "http://10.132.0.250:15021/healthz/ready": read tcp 10.132.0.2:56434->10.132.0.250:15021: read: connection reset by peer 27m Normal Started pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Container started 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-33d44f Scaled up replica set payload-processing-e2e-ipp-b-33d44f-694fc9d78d from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-33d44f Scaled up replica set payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5 from 0 to 1 27m Normal Pulled pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Container created 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-33d44f-694fc9d78d Created pod: payload-processing-e2e-ipp-b-33d44f-694fc9d78d-wc2mq 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-gzpqt Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5 Error creating: pods "payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-33d44f: serviceaccount "payload-processing-e2e-ipp-b-33d44f" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5 Created pod: payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-gzpqt 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-gzpqt Add eth0 [10.133.0.227/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-33d44f-694fc9d78d Error creating: pods "payload-processing-e2e-ipp-b-33d44f-694fc9d78d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-33d44f: serviceaccount "payload-processing-e2e-ipp-b-33d44f" not found 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-gzpqt Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-gzpqt Container started 27m Normal Created pod/payload-processing-e2e-ipp-b-33d44f-694fc9d78d-wc2mq Container created 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-33d44f-694fc9d78d-wc2mq Add eth0 [10.134.0.209/23] from ovn-kubernetes 27m Normal Started pod/payload-processing-e2e-ipp-b-33d44f-694fc9d78d-wc2mq Container started 27m Normal Pulled pod/payload-processing-e2e-ipp-b-33d44f-694fc9d78d-wc2mq Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-33d44f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-33d44f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning ProbeError pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Startup probe error: Get "http://10.132.0.250:15021/healthz/ready": dial tcp 10.132.0.250:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Startup probe failed: Get "http://10.132.0.250:15021/healthz/ready": dial tcp 10.132.0.250:15021: connect: connection refused 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-fa97ea-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-fa97ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-fa97ea Scaled up replica set payload-processing-e2e-ipp-a-fa97ea-7748477fb4 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-fa97ea Scaled up replica set payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975 Created pod: payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-fa97ea-7748477fb4 Error creating: pods "payload-processing-e2e-ipp-a-fa97ea-7748477fb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-fa97ea: serviceaccount "payload-processing-e2e-ipp-a-fa97ea" not found 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 Add eth0 [10.133.0.228/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975 Error creating: pods "payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-fa97ea: serviceaccount "payload-processing-e2e-ipp-a-fa97ea" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-fa97ea-7748477fb4 Created pod: payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv 26m Normal Started pod/payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv Container started 26m Normal Created pod/payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv Container created 26m Normal Pulled pod/payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv Add eth0 [10.133.0.229/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 Container started 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 Container created 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-fa97ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-fa97ea-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/e2e-worker-w4-0d8bae-openshift-default-6c96b68449-tgxhh Stopping container istio-proxy 26m Normal AddedLabel gateway/e2e-ipp-b-8d251f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-8d251f 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7 Created pod: e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-8d251f-openshift-default Scaled up replica set e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7 from 0 to 1 26m Normal AddedInterface pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Add eth0 [10.134.0.210/23] from ovn-kubernetes 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-8d251f Scaled up replica set payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f from 0 to 1 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f Error creating: pods "payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8d251f: serviceaccount "payload-processing-e2e-ipp-b-8d251f" not found 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-8d251f Scaled up replica set payload-processing-e2e-ipp-b-8d251f-757d49dcf5 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-8d251f-757d49dcf5 Created pod: payload-processing-e2e-ipp-b-8d251f-757d49dcf5-8lzp7 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-8d251f-757d49dcf5 Error creating: pods "payload-processing-e2e-ipp-b-8d251f-757d49dcf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8d251f: serviceaccount "payload-processing-e2e-ipp-b-8d251f" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f Created pod: payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-9fnjk 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-9fnjk Container created 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-9fnjk Container started 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-8d251f-757d49dcf5-8lzp7 Add eth0 [10.133.0.230/23] from ovn-kubernetes 26m Normal Pulled pod/payload-processing-e2e-ipp-b-8d251f-757d49dcf5-8lzp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-processing-e2e-ipp-b-8d251f-757d49dcf5-8lzp7 Container created 26m Normal Started pod/payload-processing-e2e-ipp-b-8d251f-757d49dcf5-8lzp7 Container started 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-9fnjk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-9fnjk Add eth0 [10.134.0.211/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Container started 26m Normal Created pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Container created 26m Normal Killing pod/payload-processing-e2e-ipp-a-e41918-766c7f5686-8mf57 Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e41918-7dffbd4fbc-2ln6h Stopping container payload-pre-processing 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-d68696-664fd77cb7-bwqn2 Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-b-d68696-884f5657-t256p Stopping container payload-processing 26m Warning ProbeError pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Startup probe error: Get "http://10.132.1.12:15021/healthz/ready": dial tcp 10.132.1.12:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Startup probe failed: Get "http://10.132.1.12:15021/healthz/ready": dial tcp 10.132.1.12:15021: connect: connection refused 26m Normal Killing pod/e2e-ipp-a-e41918-openshift-default-5ccdddb467-7z8gq Stopping container istio-proxy 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-d68696-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-d68696-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-d68696-5d76bd94f8 Created pod: payload-processing-e2e-ipp-b-d68696-5d76bd94f8-t4rlr 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-d68696-5d76bd94f8 Error creating: pods "payload-processing-e2e-ipp-b-d68696-5d76bd94f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d68696: serviceaccount "payload-processing-e2e-ipp-b-d68696" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-d68696 Scaled up replica set payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775 Created pod: payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-ldrrd 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-d68696 Scaled up replica set payload-processing-e2e-ipp-b-d68696-5d76bd94f8 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775 Error creating: pods "payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d68696: serviceaccount "payload-processing-e2e-ipp-b-d68696" not found 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-ldrrd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-ldrrd Add eth0 [10.133.0.235/23] from ovn-kubernetes 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-ldrrd Container created 25m Normal Started pod/payload-processing-e2e-ipp-b-d68696-5d76bd94f8-t4rlr Container started 25m Normal Created pod/payload-processing-e2e-ipp-b-d68696-5d76bd94f8-t4rlr Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-b-d68696-5d76bd94f8-t4rlr Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-d68696-5d76bd94f8-t4rlr Add eth0 [10.133.0.236/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-ldrrd Container started 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-d68696-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-d68696-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-36af31-openshift-default-74f58f595f Created pod: e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-36af31-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-36af31-openshift-default Scaled up replica set e2e-shared-a-w2-36af31-openshift-default-74f58f595f from 2 to 3 25m Normal Created pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Container created 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c Created pod: e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg 25m Normal Pulled pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Add eth0 [10.133.0.237/23] from ovn-kubernetes 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-ada0c1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal Started pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Container started 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-ada0c1-openshift-default Scaled up replica set e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c from 1 to 2 25m Normal Pulled pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Container created 25m Normal Started pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Container started 25m Normal AddedInterface pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Add eth0 [10.134.0.212/23] from ovn-kubernetes 25m Warning ProbeError pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Startup probe error: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Startup probe failed: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Startup probe error: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Startup probe failed: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused 25m Normal Killing pod/payload-pre-processing-e2e-ipp-b-33d44f-5c9c676cc5-gzpqt Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-ipp-b-33d44f-694fc9d78d-wc2mq Stopping container payload-processing 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-3c90d8-openshift-default Scaled up replica set e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99 from 2 to 3 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-3c90d8-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz Add eth0 [10.133.0.238/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99 Created pod: e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz 25m Normal Created pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz Container created 25m Normal Started pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz Container started 25m Normal Killing pod/e2e-ipp-b-33d44f-openshift-default-55d7454777-ckgx2 Stopping container istio-proxy 25m Warning Unhealthy pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz Startup probe failed: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz Startup probe error: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused... 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6886b96f9 from 3 to 2 25m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-vd2tg 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-498580-openshift-default New size: 2; reason: All metrics below target 25m Normal Killing pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-498580-openshift-default-b66db7999 Deleted pod: e2e-shared-a-w0-498580-openshift-default-b66db7999-l4rtl 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-498580-openshift-default Scaled down replica set e2e-shared-a-w0-498580-openshift-default-b66db7999 from 3 to 2 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-vd2tg Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal Killing pod/payload-pre-processing-e2e-ipp-b-8d251f-69bcff9f-9fnjk Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-ipp-b-8d251f-757d49dcf5-8lzp7 Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj Stopping container payload-processing 24m Normal Pulled pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Container created 24m Normal Started pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Container started 24m Warning Unhealthy pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Startup probe failed: Get "http://10.134.0.210:15021/healthz/ready": dial tcp 10.134.0.210:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Startup probe error: Get "http://10.134.0.210:15021/healthz/ready": dial tcp 10.134.0.210:15021: connect: connection refused... 24m Normal ScalingReplicaSet deployment/e2e-ipp-a-114c95-openshift-default Scaled up replica set e2e-ipp-a-114c95-openshift-default-68d89757b9 from 0 to 1 24m Normal SuccessfulCreate replicaset/e2e-ipp-a-114c95-openshift-default-68d89757b9 Created pod: e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp 24m Normal AddedLabel gateway/e2e-ipp-a-114c95 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-114c95 24m Normal AddedInterface pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Add eth0 [10.134.0.213/23] from ovn-kubernetes 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-476a67-fcfc97bf9-6xmwl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-476a67-6b894cfb5c-vljnj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-114c95 Scaled up replica set payload-processing-e2e-ipp-a-114c95-795d94c84c from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-114c95-795d94c84c Error creating: pods "payload-processing-e2e-ipp-a-114c95-795d94c84c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-114c95: serviceaccount "payload-processing-e2e-ipp-a-114c95" not found 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-qbn4f Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-qbn4f Container created 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-114c95-795d94c84c Created pod: payload-processing-e2e-ipp-a-114c95-795d94c84c-gsbds 24m Normal Started pod/payload-processing-e2e-ipp-a-114c95-795d94c84c-gsbds Container started 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf Error creating: pods "payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-114c95: serviceaccount "payload-processing-e2e-ipp-a-114c95" not found 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-qbn4f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-qbn4f Add eth0 [10.134.0.215/23] from ovn-kubernetes 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf Created pod: payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-qbn4f 24m Normal Created pod/payload-processing-e2e-ipp-a-114c95-795d94c84c-gsbds Container created 24m Normal Pulled pod/payload-processing-e2e-ipp-a-114c95-795d94c84c-gsbds Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-114c95-795d94c84c-gsbds Add eth0 [10.134.0.214/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-114c95 Scaled up replica set payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf from 0 to 1 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-8d251f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-8d251f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-8d251f Scaled up replica set payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-8d251f Scaled up replica set payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87 from 0 to 1 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87 Error creating: pods "payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8d251f: serviceaccount "payload-processing-e2e-ipp-b-8d251f" not found 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87 Created pod: payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-fddcb 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b Created pod: payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-9l7xc 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-99a5c2-openshift-default Scaled down replica set e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99 from 2 to 1 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-99a5c2-openshift-default New size: 1; reason: All metrics below target 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b Error creating: pods "payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8d251f: serviceaccount "payload-processing-e2e-ipp-b-8d251f" not found 24m Normal Killing pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99 Deleted pod: e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-fddcb Container started 24m Normal Created pod/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-9l7xc Container created 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-fddcb Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-fddcb Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-fddcb Add eth0 [10.132.1.31/23] from ovn-kubernetes 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-9l7xc Add eth0 [10.134.0.216/23] from ovn-kubernetes 24m Normal Started pod/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-9l7xc Container started 24m Normal Pulled pod/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-9l7xc Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Warning ProbeError pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-4w8jt Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-8d251f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-8d251f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c Stopping container payload-processing 24m Normal Killing pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Stopping container istio-proxy 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-114c95-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-114c95-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning Unhealthy pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-shared-a-w1-476a67-openshift-default-9bfcb45d7-ckgxz Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 Stopping container payload-processing 24m Normal ScalingReplicaSet deployment/e2e-ipp-a-4207cd-openshift-default Scaled up replica set e2e-ipp-a-4207cd-openshift-default-58cb5b4b55 from 1 to 2 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-4207cd-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 24m Normal SuccessfulCreate replicaset/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55 Created pod: e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv 24m Normal AddedInterface pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv Add eth0 [10.134.0.217/23] from ovn-kubernetes 24m Normal Started pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv Container started 24m Normal Created pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv Container created 24m Normal Pulled pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-498580-58d8d4685f-xpfv8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-498580-86c7764565-fxm8c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv Startup probe failed: Get "http://10.134.0.217:15021/healthz/ready": dial tcp 10.134.0.217:15021: connect: connection refused 23m Warning ProbeError pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv Startup probe error: Get "http://10.134.0.217:15021/healthz/ready": dial tcp 10.134.0.217:15021: connect: connection refused... 23m Normal Killing pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-4pff5 Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-a-w0-498580-openshift-default-b66db7999-sp5kj Stopping container istio-proxy 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-b8fd5a-75ddf6c547-kdn6t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-b8fd5a-65bdff79c-kwn97 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk Stopping container payload-processing 23m Normal Killing pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-89h5w Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-b-w1-b8fd5a-openshift-default-f9bcbdd7-fpbk7 Stopping container istio-proxy 23m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 Readiness probe failed: dial tcp 10.133.0.228:9004: connect: connection refused 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-fa97ea-67bbf6f975-f8nm8 Stopping container payload-pre-processing 23m Warning Unhealthy pod/payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv Readiness probe failed: dial tcp 10.133.0.229:9004: connect: connection refused 23m Normal Killing pod/payload-processing-e2e-ipp-a-fa97ea-7748477fb4-gd8kv Stopping container payload-processing 23m Normal Killing pod/e2e-ipp-a-fa97ea-openshift-default-6b6c5d4f7-kpfn4 Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj Stopping container payload-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-114c95-795d94c84c-gsbds Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-114c95-5ff6f64bcf-qbn4f Stopping container payload-pre-processing 23m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-3c90d8-5dbcb4889c-vd2hk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-3c90d8-b9cc8fff-nm2dt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Readiness probe failed: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Warning ProbeError pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Readiness probe error: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Normal Created pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Container created 23m Normal Pulled pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Container started 23m Normal Killing pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-pztrt Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-8rn7r Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-b-w0-3c90d8-openshift-default-b8c68db99-gqkcz Stopping container istio-proxy 23m Warning Unhealthy pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Startup probe failed: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused 23m Warning ProbeError pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Startup probe error: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused... 22m Warning Unhealthy pod/payload-processing-e2e-worker-w1-b024b5-54459f766c-r5fpj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-b024b5-59c565544c-f7gfl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 Stopping container payload-pre-processing 22m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687 Deleted pod: e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-bcdb4e-openshift-default Scaled down replica set e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687 from 2 to 1 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-bcdb4e-openshift-default New size: 1; reason: All metrics below target 22m Normal Killing pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-b674s Stopping container istio-proxy 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-114c95-7c5799bc88 Error creating: pods "payload-processing-e2e-ipp-a-114c95-7c5799bc88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-114c95: serviceaccount "payload-processing-e2e-ipp-a-114c95" not found 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95 Error creating: pods "payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-114c95: serviceaccount "payload-processing-e2e-ipp-a-114c95" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95 Created pod: payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-hkk4k 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-114c95 Scaled up replica set payload-processing-e2e-ipp-a-114c95-7c5799bc88 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-114c95 Scaled up replica set payload-pre-processing-e2e-ipp-a-114c95-7c7856f95 from 0 to 1 22m Normal Pulled pod/payload-processing-e2e-ipp-a-114c95-7c5799bc88-w2b2g Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Started pod/payload-processing-e2e-ipp-a-114c95-7c5799bc88-w2b2g Container started 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-114c95-7c5799bc88 Created pod: payload-processing-e2e-ipp-a-114c95-7c5799bc88-w2b2g 22m Normal Created pod/payload-processing-e2e-ipp-a-114c95-7c5799bc88-w2b2g Container created 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-114c95-7c5799bc88-w2b2g Add eth0 [10.133.0.241/23] from ovn-kubernetes 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-hkk4k Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-hkk4k Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-hkk4k Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-hkk4k Add eth0 [10.133.0.240/23] from ovn-kubernetes 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-114c95-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-114c95-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-714f8f-openshift-default Scaled down replica set e2e-shared-b-w6-714f8f-openshift-default-85496857c7 from 2 to 1 22m Normal Killing pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c Stopping container istio-proxy 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-714f8f-openshift-default New size: 1; reason: All metrics below target 22m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-714f8f-openshift-default-85496857c7 Deleted pod: e2e-shared-b-w6-714f8f-openshift-default-85496857c7-cdl2c 22m Normal Killing pod/payload-processing-e2e-ipp-b-8d251f-7fb56dfb5b-9l7xc Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-8d251f-7847b9ff87-fddcb Stopping container payload-pre-processing 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-671959-86968d9675-tpkw6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-processing-e2e-worker-w0-671959-6bb586d9d5-zwlhm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-328d61-openshift-default-7857fc774c Deleted pod: e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-328d61-openshift-default New size: 1; reason: All metrics below target 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-328d61-openshift-default Scaled down replica set e2e-shared-b-w5-328d61-openshift-default-7857fc774c from 2 to 1 22m Normal Killing pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-g49ds Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Stopping container istio-proxy 22m Normal AddedLabel gateway/e2e-ipp-b-8f0029 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-8f0029 22m Warning ProbeError pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-ipp-b-8d251f-openshift-default-7c4f9bdf7-dz6kr Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal ScalingReplicaSet deployment/e2e-ipp-b-8f0029-openshift-default Scaled up replica set e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c from 0 to 1 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb 22m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb Add eth0 [10.134.0.219/23] from ovn-kubernetes 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf Error creating: pods "payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8f0029: serviceaccount "payload-processing-e2e-ipp-b-8f0029" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-8f0029 Scaled up replica set payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879 from 0 to 1 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879 Error creating: pods "payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8f0029: serviceaccount "payload-processing-e2e-ipp-b-8f0029" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879 Created pod: payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-x87r2 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-8f0029 Scaled up replica set payload-processing-e2e-ipp-b-8f0029-7497dc5fdf from 0 to 1 22m Normal Created pod/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-r6vv5 Container created 22m Normal Started pod/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-r6vv5 Container started 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-r6vv5 Add eth0 [10.134.0.221/23] from ovn-kubernetes 22m Normal Pulled pod/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-r6vv5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf Created pod: payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-r6vv5 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-x87r2 Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-x87r2 Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-x87r2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-x87r2 Add eth0 [10.134.0.220/23] from ovn-kubernetes 22m Normal Killing pod/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-a-4207cd-55648f6b98-x88sq Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-4207cd-769675f9dc-ndr4b Stopping container payload-pre-processing 22m Normal AddedInterface pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Add eth0 [10.133.0.242/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c Created pod: e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j 22m Normal Started pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Container started 22m Normal Created pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Container created 22m Normal Pulled pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-ada0c1-openshift-default Scaled up replica set e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c from 2 to 3 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-ada0c1-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 22m Warning Unhealthy pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Startup probe failed: Get "http://10.133.0.242:15021/healthz/ready": dial tcp 10.133.0.242:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Startup probe error: Get "http://10.133.0.242:15021/healthz/ready": dial tcp 10.133.0.242:15021: connect: connection refused... 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-d10721-openshift-default Scaled up replica set e2e-shared-a-w3-d10721-openshift-default-bf897765f from 1 to 2 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-d10721-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-d10721-openshift-default-bf897765f Created pod: e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 22m Normal Pulled pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 Add eth0 [10.133.0.243/23] from ovn-kubernetes 22m Normal Started pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 Container started 22m Normal Created pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 Container created 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-bcdb4e-5497455cf-rnptb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-7fxkv Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Stopping container istio-proxy 22m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-bcdb4e-7db995c565-6fvlh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning ProbeError pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-worker-w1-b024b5-openshift-default-75d4b8c46f-xtjwq Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning Unhealthy pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 Startup probe failed: Get "http://10.133.0.243:15021/healthz/ready": dial tcp 10.133.0.243:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 Startup probe error: Get "http://10.133.0.243:15021/healthz/ready": dial tcp 10.133.0.243:15021: connect: connection refused... 22m Normal Killing pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-shared-a-w6-bcdb4e-openshift-default-5f88bc4687-x8zkf Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal Killing pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Stopping container istio-proxy 21m Warning ProbeError pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-worker-w0-671959-openshift-default-586d4c9cc5-mdw5x Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Killing pod/payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx Stopping container payload-pre-processing 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg 21m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg Add eth0 [10.132.1.48/23] from ovn-kubernetes 21m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q Add eth0 [10.133.0.245/23] from ovn-kubernetes 21m Normal ScalingReplicaSet deployment/e2e-ipp-b-8f0029-openshift-default Scaled up replica set e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c from 1 to 4 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-8f0029-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-8f0029-59cccc7879-x87r2 Stopping container payload-pre-processing 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q 21m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s Add eth0 [10.134.0.222/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s 21m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb Container started 21m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Killing pod/payload-processing-e2e-ipp-b-8f0029-7497dc5fdf-r6vv5 Stopping container payload-processing 21m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb Container created 21m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s Container created 21m Normal Killing pod/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl Stopping container payload-processing 21m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q Container started 21m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s Container started 21m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x Stopping container payload-pre-processing 21m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q Container created 21m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg Container created 21m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg Container started 21m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-714f8f-694d787d6c-4tbbx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-714f8f-85559d789f-vhc4x Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s Startup probe failed: Get "http://10.134.0.222:15021/healthz/ready": dial tcp 10.134.0.222:15021: connect: connection refused 21m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg Startup probe failed: Get "http://10.132.1.48:15021/healthz/ready": dial tcp 10.132.1.48:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg Startup probe error: Get "http://10.132.1.48:15021/healthz/ready": dial tcp 10.132.1.48:15021: connect: connection refused... 21m Normal Killing pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q Startup probe failed: Get "http://10.133.0.245:15021/healthz/ready": dial tcp 10.133.0.245:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s Startup probe error: Get "http://10.134.0.222:15021/healthz/ready": dial tcp 10.134.0.222:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb Startup probe failed: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q Startup probe error: Get "http://10.133.0.245:15021/healthz/ready": dial tcp 10.133.0.245:15021: connect: connection refused... 21m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb Startup probe error: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused... 21m Normal Killing pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-5pgtv Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-shared-b-w6-714f8f-openshift-default-85496857c7-nmddh Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning ProbeError pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-4207cd-openshift-default-58cb5b4b55-h7ntn Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-8f0029-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-8f0029-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal Killing pod/payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w Stopping container payload-pre-processing 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-8f0029 Scaled up replica set payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f from 0 to 1 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-d10721-6d9947dc59-xfh7x Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-d10721-7cb984c4cc-jtcsl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-8f0029 Scaled up replica set payload-processing-e2e-ipp-b-8f0029-5b558588db from 0 to 1 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f Error creating: pods "payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8f0029: serviceaccount "payload-processing-e2e-ipp-b-8f0029" not found 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f Created pod: payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-7tcms 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-8f0029-5b558588db Error creating: pods "payload-processing-e2e-ipp-b-8f0029-5b558588db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8f0029: serviceaccount "payload-processing-e2e-ipp-b-8f0029" not found 21m Normal AddedInterface pod/payload-processing-e2e-ipp-b-8f0029-5b558588db-d5gpn Add eth0 [10.134.0.224/23] from ovn-kubernetes 21m Normal Created pod/payload-processing-e2e-ipp-b-8f0029-5b558588db-d5gpn Container created 21m Normal Started pod/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-7tcms Container started 21m Normal Created pod/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-7tcms Container created 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-7tcms Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-7tcms Add eth0 [10.134.0.223/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-8f0029-5b558588db Created pod: payload-processing-e2e-ipp-b-8f0029-5b558588db-d5gpn 21m Normal Started pod/payload-processing-e2e-ipp-b-8f0029-5b558588db-d5gpn Container started 21m Normal Pulled pod/payload-processing-e2e-ipp-b-8f0029-5b558588db-d5gpn Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-8f0029-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-8f0029-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Normal Killing pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-gt67s Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w3-d10721-openshift-default-bf897765f-pslz7 Stopping container istio-proxy 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-be301b-57cdb5dfc9-p264w Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-worker-w6-be301b-77c7dbd46f-wfqh9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-d68696-5c9bd48775-ldrrd Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-d68696-5d76bd94f8-t4rlr Stopping container payload-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc Stopping container payload-processing 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 20m Normal ScalingReplicaSet deployment/e2e-ipp-b-8f0029-openshift-default Scaled up replica set e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c from 4 to 8 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-8f0029-openshift-default New size: 8; reason: All metrics below target 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb Stopping container payload-pre-processing 20m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg Container created 20m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 Container started 20m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 Container created 20m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg Add eth0 [10.132.1.55/23] from ovn-kubernetes 20m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc Add eth0 [10.134.0.225/23] from ovn-kubernetes 20m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 Container started 20m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg Container started 20m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 Container created 20m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc Container created 20m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc Container started 20m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 Add eth0 [10.132.1.56/23] from ovn-kubernetes 20m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 Add eth0 [10.133.0.246/23] from ovn-kubernetes 20m Normal Killing pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Stopping container istio-proxy 20m Warning ProbeError pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-ipp-b-d68696-openshift-default-6dd844fcf-t84vf Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 Startup probe failed: Get "http://10.132.1.56:15021/healthz/ready": dial tcp 10.132.1.56:15021: connect: connection refused 20m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc Startup probe error: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused... 20m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 Startup probe error: Get "http://10.132.1.56:15021/healthz/ready": dial tcp 10.132.1.56:15021: connect: connection refused... 20m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 Startup probe error: Get "http://10.133.0.246:15021/healthz/ready": dial tcp 10.133.0.246:15021: connect: connection refused... 20m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg Startup probe error: Get "http://10.132.1.55:15021/healthz/ready": dial tcp 10.132.1.55:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 Startup probe failed: Get "http://10.133.0.246:15021/healthz/ready": dial tcp 10.133.0.246:15021: connect: connection refused 20m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc Startup probe failed: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused 20m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg Startup probe failed: Get "http://10.132.1.55:15021/healthz/ready": dial tcp 10.132.1.55:15021: connect: connection refused 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-99a5c2-b794bc94c-nvpzl Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr Stopping container payload-processing 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-8f0029-openshift-default New size: 10; reason: All metrics below target 20m Normal ScalingReplicaSet deployment/e2e-ipp-b-8f0029-openshift-default Scaled up replica set e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c from 8 to 10 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c (combined from similar events): Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c Created pod: e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm 20m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 20m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 20m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-sgntd 20m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-sgntd Stopping container discovery 20m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm Add eth0 [10.133.0.247/23] from ovn-kubernetes 20m Normal AddedInterface pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 Add eth0 [10.132.1.58/23] from ovn-kubernetes 20m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 Container created 20m Normal Pulled pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Created pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm Container created 20m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm Container started 20m Normal Started pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 Container started 20m Normal Killing pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-n474b Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Stopping container istio-proxy 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-99a5c2-7654644d6f-8ndnr Readiness probe failed: timeout: failed to connect service "10.133.0.185:50052" within 1s: context deadline exceeded 20m Normal Killing pod/e2e-worker-w6-be301b-openshift-default-7bd4946d5f-z6hx5 Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-ada0c1-7548dd6f85-hkqgc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning ProbeError pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-knmhg Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-ada0c1-6fd9b86dc5-j8pzb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 Startup probe failed: Get "http://10.132.1.58:15021/healthz/ready": dial tcp 10.132.1.58:15021: connect: connection refused 20m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 Startup probe error: Get "http://10.132.1.58:15021/healthz/ready": dial tcp 10.132.1.58:15021: connect: connection refused... 20m Warning ProbeError pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-b-w3-ada0c1-openshift-default-79866ddb6c-tfh7j Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm Startup probe error: Get "http://10.133.0.247:15021/healthz/ready": dial tcp 10.133.0.247:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm Startup probe failed: Get "http://10.133.0.247:15021/healthz/ready": dial tcp 10.133.0.247:15021: connect: connection refused 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-114c95-7c7856f95-hkk4k Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-a-114c95-7c5799bc88-w2b2g Stopping container payload-processing 20m Normal Killing pod/payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f Stopping container payload-pre-processing 20m Normal Killing pod/e2e-ipp-a-114c95-openshift-default-68d89757b9-nxktp Stopping container istio-proxy 20m Warning ProbeError pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/e2e-shared-a-w5-99a5c2-openshift-default-6d6895cd99-mn4mg Stopping container istio-proxy 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-8f0029-7fdf69d77f-7tcms Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-8f0029-5b558588db-d5gpn Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc Stopping container payload-pre-processing 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-daaf26-565c84fdcf-fq96f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw Stopping container payload-processing 19m Warning Unhealthy pod/payload-processing-e2e-worker-w3-daaf26-86c9494789-zkrtm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-mstv7 Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-pb5cb Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-z648q Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-9j9s9 Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-s5vzg Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-tg7l5 Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-vhs2s Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-6gfxg Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ppjcm Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-8f0029-openshift-default-dd4bb5c6c-ntkbc Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w3-daaf26-openshift-default-64f4dbcc55-h8kwb Stopping container istio-proxy 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-328d61-74b757865c-zqvlw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-328d61-5d597c4b5f-6sfgc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b Stopping container payload-pre-processing 19m Warning Unhealthy pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-b-w5-328d61-openshift-default-7857fc774c-sml6w Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-zkrnf Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Stopping container istio-proxy 19m Normal Killing pod/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl Stopping container payload-processing 19m Warning Unhealthy pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-l8fgx Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-a-w2-36af31-openshift-default-74f58f595f-7x6vd Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-36af31-5f69cb665f-tlc8b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf Stopping container payload-processing 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-36af31-868779fbd5-cqsxb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-worker-w5-c9e377-5b9f9bd69c-xdrrl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-c9e377-67f55d5979-774ph Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w5-c9e377-openshift-default-bbb7b4499-lvxjl Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-b-w2-f2bdf3-openshift-default-cf4c47767-mspbv Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/payload-processing-e2e-worker-w2-735123-8466d8d68f-ssg4w Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 Stopping container payload-pre-processing 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-f2bdf3-7bf45d675f-8pf25 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-f2bdf3-68f945d689-pqxbf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-735123-6bd8bf8799-x9zr5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-worker-w2-735123-openshift-default-5889966f5-cddfp Stopping container istio-proxy 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-ldf84 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 2 to 4 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-zrjvf 17m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-zrjvf Container created 17m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-zrjvf Container started 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Add eth0 [10.134.0.232/23] from ovn-kubernetes 17m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-zrjvf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-zrjvf Add eth0 [10.133.0.249/23] from ovn-kubernetes 17m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Container started 17m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Container created 17m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-zrjvf Startup probe failed: Get "http://10.133.0.249:15021/healthz/ready": dial tcp 10.133.0.249:15021: connect: connection refused 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-zrjvf Startup probe error: Get "http://10.133.0.249:15021/healthz/ready": dial tcp 10.133.0.249:15021: connect: connection refused... 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Startup probe error: Get "http://10.134.0.232:15021/healthz/ready": dial tcp 10.134.0.232:15021: connect: connection refused... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Startup probe failed: Get "http://10.134.0.232:15021/healthz/ready": dial tcp 10.134.0.232:15021: connect: connection refused 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 4 to 5 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-6bxb6 17m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Container created 17m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Container started 17m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Add eth0 [10.134.0.233/23] from ovn-kubernetes 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Startup probe error: Get "http://10.134.0.233:15021/healthz/ready": dial tcp 10.134.0.233:15021: connect: connection refused... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Startup probe failed: Get "http://10.134.0.233:15021/healthz/ready": dial tcp 10.134.0.233:15021: connect: connection refused 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 5 to 8 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-8gm8p 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-nqx6q 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-hlnsl 16m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Container started 16m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Container created 16m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Add eth0 [10.134.0.235/23] from ovn-kubernetes 16m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Container started 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Add eth0 [10.133.0.251/23] from ovn-kubernetes 16m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Container created 16m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Container started 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Add eth0 [10.132.1.67/23] from ovn-kubernetes 16m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Container created 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Startup probe failed: Get "http://10.134.0.235:15021/healthz/ready": dial tcp 10.134.0.235:15021: connect: connection refused 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Startup probe failed: Get "http://10.133.0.251:15021/healthz/ready": dial tcp 10.133.0.251:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Startup probe error: Get "http://10.133.0.251:15021/healthz/ready": dial tcp 10.133.0.251:15021: connect: connection refused... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Startup probe error: Get "http://10.134.0.235:15021/healthz/ready": dial tcp 10.134.0.235:15021: connect: connection refused... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Startup probe error: Get "http://10.132.1.67:15021/healthz/ready": dial tcp 10.132.1.67:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Startup probe failed: Get "http://10.132.1.67:15021/healthz/ready": dial tcp 10.132.1.67:15021: connect: connection refused 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Startup probe failed: Get "http://10.133.0.251:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Startup probe failed: Get "http://10.134.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Startup probe error: Get "http://10.134.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Startup probe error: Get "http://10.133.0.251:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Startup probe failed: Get "http://10.132.1.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-8gm8p Startup probe error: Get "http://10.132.1.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-qq9v6 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 8 to 10 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-zhprr 16m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-zhprr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Add eth0 [10.134.0.237/23] from ovn-kubernetes 16m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-zhprr Container started 16m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-zhprr Container created 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-zhprr Add eth0 [10.132.1.68/23] from ovn-kubernetes 16m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Container started 16m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Container created 16m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-zhprr Startup probe failed: Get "http://10.132.1.68:15021/healthz/ready": dial tcp 10.132.1.68:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-zhprr Startup probe error: Get "http://10.132.1.68:15021/healthz/ready": dial tcp 10.132.1.68:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Startup probe failed: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Startup probe error: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Readiness probe error: Get "http://10.133.0.251:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Readiness probe failed: Get "http://10.134.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-hlnsl Readiness probe failed: Get "http://10.133.0.251:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-nqx6q Readiness probe error: Get "http://10.134.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Normal AddedLabel gateway/e2e-worker-single-main-e8e612 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-e8e612 12m Normal SuccessfulCreate replicaset/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5 Created pod: e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj 12m Normal ScalingReplicaSet deployment/e2e-worker-single-main-e8e612-openshift-default Scaled up replica set e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5 from 0 to 1 12m Normal AddedInterface pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Add eth0 [10.133.0.252/23] from ovn-kubernetes 12m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-e8e612-77c78984f5 Error creating: pods "payload-pre-processing-e2e-worker-single-main-e8e612-77c78984f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-e8e612: serviceaccount "payload-processing-e2e-worker-single-main-e8e612" not found 12m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-e8e612-59fc89c957 Error creating: pods "payload-processing-e2e-worker-single-main-e8e612-59fc89c957-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-e8e612: serviceaccount "payload-processing-e2e-worker-single-main-e8e612" not found 12m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-e8e612-77c78984f5 Created pod: payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k 12m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-e8e612-59fc89c957 Created pod: payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 12m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-e8e612 Scaled up replica set payload-processing-e2e-worker-single-main-e8e612-59fc89c957 from 0 to 1 12m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-e8e612 Scaled up replica set payload-pre-processing-e2e-worker-single-main-e8e612-77c78984f5 from 0 to 1 12m Normal Created pod/payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k Container created 12m Normal Started pod/payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k Container started 12m Normal Started pod/payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 Container started 12m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k Add eth0 [10.133.0.253/23] from ovn-kubernetes 12m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 Add eth0 [10.133.0.254/23] from ovn-kubernetes 12m Normal Created pod/payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 Container created 12m Normal Pulled pod/payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal Pulled pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Warning ProbeError pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Readiness probe error: Get "http://10.133.0.252:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Readiness probe failed: Get "http://10.133.0.252:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Normal Created pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Container created 12m Normal Started pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Container started 12m Warning ProbeError pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Startup probe error: Get "http://10.133.0.252:15021/healthz/ready": dial tcp 10.133.0.252:15021: connect: connection refused... 12m Warning Unhealthy pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Startup probe failed: Get "http://10.133.0.252:15021/healthz/ready": dial tcp 10.133.0.252:15021: connect: connection refused 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-e8e612-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-e8e612-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Normal Killing pod/payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 Stopping container payload-processing 11m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k Stopping container payload-pre-processing 11m Normal Killing pod/e2e-worker-single-main-e8e612-openshift-default-585fb6ccd5xdkmj Stopping container istio-proxy 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-e8e612-59fc89c95x8pm2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-e8e612-77c78zv22k Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Readiness probe error: Get "http://10.134.0.233:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-6bxb6 Readiness probe failed: Get "http://10.134.0.233:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 10m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Readiness probe error: Get "http://10.134.0.237:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Readiness probe failed: Get "http://10.134.0.232:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 10m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-ldf84 Readiness probe error: Get "http://10.134.0.232:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-qq9v6 Readiness probe failed: Get "http://10.134.0.237:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m38s Normal SuccessfulCreate replicaset/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5 Created pod: e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv 8m38s Normal AddedLabel gateway/e2e-worker-single-main-2e17bf Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-2e17bf 8m38s Normal ScalingReplicaSet deployment/e2e-worker-single-main-2e17bf-openshift-default Scaled up replica set e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5 from 0 to 1 8m38s Normal AddedInterface pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Add eth0 [10.133.0.255/23] from ovn-kubernetes 8m33s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-2e17bf Scaled up replica set payload-processing-e2e-worker-single-main-2e17bf-cfc884b45 from 0 to 1 8m33s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-2e17bf Scaled up replica set payload-pre-processing-e2e-worker-single-main-2e17bf-b8894ffb7 from 0 to 1 8m33s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-2e17bf-cfc884b45 Created pod: payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc 8m33s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894ffb7 Created pod: payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg 8m33s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-2e17bf-cfc884b45 Error creating: pods "payload-processing-e2e-worker-single-main-2e17bf-cfc884b45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2e17bf: serviceaccount "payload-processing-e2e-worker-single-main-2e17bf" not found 8m33s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894ffb7 Error creating: pods "payload-pre-processing-e2e-worker-single-main-2e17bf-b8894ffb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2e17bf: serviceaccount "payload-processing-e2e-worker-single-main-2e17bf" not found 8m32s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m32s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg Add eth0 [10.133.1.0/23] from ovn-kubernetes 8m32s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc Add eth0 [10.133.1.1/23] from ovn-kubernetes 8m32s Normal Started pod/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg Container started 8m32s Normal Created pod/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg Container created 8m32s Normal Pulled pod/payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m32s Normal Started pod/payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc Container started 8m32s Normal Created pod/payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc Container created 8m19s Warning Unhealthy pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Readiness probe failed: Get "http://10.133.0.255:15021/healthz/ready": read tcp 10.133.0.2:56866->10.133.0.255:15021: read: connection reset by peer 8m19s Warning ProbeError pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Readiness probe error: Get "http://10.133.0.255:15021/healthz/ready": read tcp 10.133.0.2:56866->10.133.0.255:15021: read: connection reset by peer... 8m19s Normal Pulled pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m19s Normal Created pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Container created 8m19s Normal Started pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Container started 8m14s Warning ProbeError pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Startup probe error: Get "http://10.133.0.255:15021/healthz/ready": dial tcp 10.133.0.255:15021: connect: connection refused... 8m14s Warning Unhealthy pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Startup probe failed: Get "http://10.133.0.255:15021/healthz/ready": dial tcp 10.133.0.255:15021: connect: connection refused 7m38s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-2e17bf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m38s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-2e17bf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m25s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg Stopping container payload-pre-processing 7m25s Normal Killing pod/payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc Stopping container payload-processing 7m17s Normal Killing pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Stopping container istio-proxy 7m13s Warning Unhealthy pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Readiness probe failed: HTTP probe failed with statuscode: 503 7m13s Warning ProbeError pod/e2e-worker-single-main-2e17bf-openshift-default-9f6bcd5d5-rcgnv Readiness probe error: HTTP probe failed with statuscode: 503... 7m6s Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-97mst Readiness probe failed: Get "http://10.132.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m6s Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-97mst Readiness probe error: Get "http://10.132.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-2e17bf-b8894mhrbg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-2e17bf-cfc884b4528nwc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m29s Normal ScalingReplicaSet deployment/e2e-ait-1c206c91-openshift-default Scaled up replica set e2e-ait-1c206c91-openshift-default-697d6f596c from 0 to 1 5m29s Normal SuccessfulCreate replicaset/e2e-ait-1c206c91-openshift-default-697d6f596c Created pod: e2e-ait-1c206c91-openshift-default-697d6f596c-272rs 5m29s Normal AddedLabel gateway/e2e-ait-1c206c91 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1c206c91 5m29s Normal AddedInterface pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Add eth0 [10.134.1.6/23] from ovn-kubernetes 5m23s Normal AddedInterface pod/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-n88l8 Add eth0 [10.133.1.5/23] from ovn-kubernetes 5m23s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-1c206c91 Scaled up replica set payload-processing-e2e-ait-1c206c91-77d45f557c from 0 to 1 5m23s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-1c206c91-77d45f557c Created pod: payload-processing-e2e-ait-1c206c91-77d45f557c-l6rhp 5m23s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc Created pod: payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-n88l8 5m23s Normal Pulled pod/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-n88l8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m23s Warning FailedCreate replicaset/payload-processing-e2e-ait-1c206c91-77d45f557c Error creating: pods "payload-processing-e2e-ait-1c206c91-77d45f557c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1c206c91: serviceaccount "payload-processing-e2e-ait-1c206c91" not found 5m23s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc Error creating: pods "payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1c206c91: serviceaccount "payload-processing-e2e-ait-1c206c91" not found 5m23s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-1c206c91 Scaled up replica set payload-pre-processing-e2e-ait-1c206c91-9f7879bdc from 0 to 1 5m22s Normal Created pod/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-n88l8 Container created 5m22s Normal Created pod/payload-processing-e2e-ait-1c206c91-77d45f557c-l6rhp Container created 5m22s Normal Pulled pod/payload-processing-e2e-ait-1c206c91-77d45f557c-l6rhp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m22s Normal AddedInterface pod/payload-processing-e2e-ait-1c206c91-77d45f557c-l6rhp Add eth0 [10.133.1.6/23] from ovn-kubernetes 5m22s Normal Started pod/payload-processing-e2e-ait-1c206c91-77d45f557c-l6rhp Container started 5m22s Normal Started pod/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-n88l8 Container started 5m10s Normal Created pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Container created 5m10s Warning Unhealthy pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Readiness probe failed: Get "http://10.134.1.6:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m10s Warning ProbeError pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Readiness probe error: Get "http://10.134.1.6:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m10s Normal Started pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Container started 5m10s Normal Pulled pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m5s Warning ProbeError pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Startup probe error: Get "http://10.134.1.6:15021/healthz/ready": dial tcp 10.134.1.6:15021: connect: connection refused... 5m5s Warning Unhealthy pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Startup probe failed: Get "http://10.134.1.6:15021/healthz/ready": dial tcp 10.134.1.6:15021: connect: connection refused 5m Normal Killing pod/payload-pre-processing-e2e-ait-1c206c91-9f7879bdc-n88l8 Stopping container payload-pre-processing 5m Normal Killing pod/payload-processing-e2e-ait-1c206c91-77d45f557c-l6rhp Stopping container payload-processing 4m59s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1c206c91-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m59s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1c206c91-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m54s Normal Killing pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Stopping container istio-proxy 4m52s Normal AddedLabel gateway/e2e-authcfg-http-aebdfd Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-aebdfd 4m52s Normal ScalingReplicaSet deployment/e2e-authcfg-http-aebdfd-openshift-default Scaled up replica set e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c from 0 to 1 4m52s Normal SuccessfulCreate replicaset/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c Created pod: e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt 4m52s Normal AddedInterface pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Add eth0 [10.133.1.7/23] from ovn-kubernetes 4m49s Warning Unhealthy pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Readiness probe failed: Get "http://10.134.1.6:15021/healthz/ready": dial tcp 10.134.1.6:15021: connect: connection refused 4m49s Warning ProbeError pod/e2e-ait-1c206c91-openshift-default-697d6f596c-272rs Readiness probe error: Get "http://10.134.1.6:15021/healthz/ready": dial tcp 10.134.1.6:15021: connect: connection refused... 4m47s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc Error creating: pods "payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-aebdfd: serviceaccount "payload-processing-e2e-authcfg-http-aebdfd" not found 4m47s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-aebdfd Scaled up replica set payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755 from 0 to 1 4m47s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc Created pod: payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-pf5zz 4m47s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-aebdfd Scaled up replica set payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc from 0 to 1 4m47s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755 Error creating: pods "payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-aebdfd: serviceaccount "payload-processing-e2e-authcfg-http-aebdfd" not found 4m47s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755 Created pod: payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-tp78s 4m46s Normal Created pod/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-tp78s Container created 4m46s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-pf5zz Add eth0 [10.133.1.8/23] from ovn-kubernetes 4m46s Normal Pulled pod/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-pf5zz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m46s Normal Created pod/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-pf5zz Container created 4m46s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-tp78s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m46s Normal Started pod/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-pf5zz Container started 4m46s Normal Started pod/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-tp78s Container started 4m46s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-tp78s Add eth0 [10.133.1.9/23] from ovn-kubernetes 4m34s Warning Unhealthy pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Readiness probe failed: Get "http://10.133.1.7:15021/healthz/ready": dial tcp 10.133.1.7:15021: connect: connection refused 4m34s Normal Started pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Container started 4m34s Normal Created pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Container created 4m34s Normal Pulled pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m34s Warning ProbeError pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Readiness probe error: Get "http://10.133.1.7:15021/healthz/ready": dial tcp 10.133.1.7:15021: connect: connection refused... 4m28s Warning Unhealthy pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Startup probe failed: Get "http://10.133.1.7:15021/healthz/ready": dial tcp 10.133.1.7:15021: connect: connection refused 4m28s Warning ProbeError pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Startup probe error: Get "http://10.133.1.7:15021/healthz/ready": dial tcp 10.133.1.7:15021: connect: connection refused... 3m52s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-aebdfd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m52s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-aebdfd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m3s Normal Killing pod/payload-processing-e2e-authcfg-http-aebdfd-c6f7cc4bc-pf5zz Stopping container payload-processing 3m3s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-aebdfd-7688679755-tp78s Stopping container payload-pre-processing 2m49s Normal Killing pod/e2e-authcfg-http-aebdfd-openshift-default-7d567cc99c-8gdrt Stopping container istio-proxy