=== events in openshift-ingress (2026-10-30T01:46:37Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 51m Normal Scheduled pod/payload-processing-5f78687d5b-nkqms Successfully assigned openshift-ingress/payload-processing-5f78687d5b-nkqms to ip-10-0-140-206.ec2.internal 27m Normal Scheduled pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Successfully assigned openshift-ingress/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh to ip-10-0-140-206.ec2.internal 22m Normal Scheduled pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm Successfully assigned openshift-ingress/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm to ip-10-0-143-7.ec2.internal 22m Normal Scheduled pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v Successfully assigned openshift-ingress/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v to ip-10-0-143-7.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-api-b-da60de-877f4976-zv2gh Successfully assigned openshift-ingress/payload-processing-e2e-api-b-da60de-877f4976-zv2gh to ip-10-0-130-169.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn to ip-10-0-143-7.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj to ip-10-0-130-169.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq to ip-10-0-130-169.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-derive-8b58192f-f48df6874-wbtnf Successfully assigned openshift-ingress/payload-processing-e2e-derive-8b58192f-f48df6874-wbtnf to ip-10-0-130-169.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp Successfully assigned openshift-ingress/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp to ip-10-0-130-169.ec2.internal 38m Normal Scheduled pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Successfully assigned openshift-ingress/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c to ip-10-0-143-7.ec2.internal 44m Normal Scheduled pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Successfully assigned openshift-ingress/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt to ip-10-0-140-206.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z to ip-10-0-130-169.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt to ip-10-0-130-169.ec2.internal 19m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg to ip-10-0-130-169.ec2.internal 19m Warning FailedScheduling pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Warning FailedScheduling pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 8m59s Normal Scheduled pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Successfully assigned openshift-ingress/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 to ip-10-0-140-206.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb to ip-10-0-140-206.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 Successfully assigned openshift-ingress/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 to ip-10-0-140-206.ec2.internal 5m32s Normal Scheduled pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Successfully assigned openshift-ingress/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms to ip-10-0-140-206.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-2686h Successfully assigned openshift-ingress/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-2686h to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx to ip-10-0-143-7.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-mt-08065c8e-68f98d88c4-svnks Successfully assigned openshift-ingress/payload-processing-e2e-mt-08065c8e-68f98d88c4-svnks to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 to ip-10-0-140-206.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Successfully assigned openshift-ingress/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 to ip-10-0-130-169.ec2.internal 44m Normal Scheduled pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Successfully assigned openshift-ingress/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl to ip-10-0-143-7.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c to ip-10-0-143-7.ec2.internal 44m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws to ip-10-0-143-7.ec2.internal 8m54s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh to ip-10-0-140-206.ec2.internal 22m Warning FailedScheduling pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww Successfully assigned openshift-ingress/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 to ip-10-0-140-206.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p to ip-10-0-140-206.ec2.internal 5m26s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g to ip-10-0-130-169.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Successfully assigned openshift-ingress/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf to ip-10-0-143-7.ec2.internal 19m Warning FailedScheduling pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Successfully assigned openshift-ingress/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b to ip-10-0-143-7.ec2.internal 39m Normal Scheduled pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Successfully assigned openshift-ingress/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp to ip-10-0-140-206.ec2.internal 42m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp to ip-10-0-130-169.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 to ip-10-0-140-206.ec2.internal 39m Normal Scheduled pod/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw Successfully assigned openshift-ingress/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw to ip-10-0-140-206.ec2.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-eaf753-8649859f-nml4p Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-eaf753-8649859f-nml4p to ip-10-0-130-169.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-b-34e896-6fbfdbbf68-cr55q Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-34e896-6fbfdbbf68-cr55q to ip-10-0-130-169.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-sc42t Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-sc42t to ip-10-0-130-169.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-sql7t Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-sql7t to ip-10-0-140-206.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 Successfully assigned openshift-ingress/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k Successfully assigned openshift-ingress/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k to ip-10-0-140-206.ec2.internal 48m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp to ip-10-0-140-206.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm to ip-10-0-140-206.ec2.internal 19m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v to ip-10-0-130-169.ec2.internal 19m Warning FailedScheduling pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 to ip-10-0-130-169.ec2.internal 22m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Successfully assigned openshift-ingress/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db to ip-10-0-143-7.ec2.internal 21m Warning FailedScheduling pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-m6bds Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-m6bds to ip-10-0-130-169.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-9vx7m Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-9vx7m to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 Successfully assigned openshift-ingress/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 to ip-10-0-143-7.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-4s82z Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-4s82z to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm to ip-10-0-140-206.ec2.internal 50m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Successfully assigned openshift-ingress/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 Successfully assigned openshift-ingress/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 to ip-10-0-130-169.ec2.internal 31m Normal Scheduled pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 Successfully assigned openshift-ingress/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 to ip-10-0-143-7.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf to ip-10-0-130-169.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z to ip-10-0-143-7.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl to ip-10-0-143-7.ec2.internal 40m Normal Scheduled pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Successfully assigned openshift-ingress/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 to ip-10-0-130-169.ec2.internal 21m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Warning FailedScheduling pod/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 35m Normal Scheduled pod/payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b to ip-10-0-130-169.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-fcnrd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-fcnrd to ip-10-0-130-169.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-b-687471-74d85f5bdd-qw568 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-687471-74d85f5bdd-qw568 to ip-10-0-130-169.ec2.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-kr794 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-kr794 to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g to ip-10-0-130-169.ec2.internal 28m Normal Scheduled pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq Successfully assigned openshift-ingress/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq to ip-10-0-140-206.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-migrate-837d7038-569cf58996-hz8jm Successfully assigned openshift-ingress/payload-processing-e2e-migrate-837d7038-569cf58996-hz8jm to ip-10-0-140-206.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l Successfully assigned openshift-ingress/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l to ip-10-0-130-169.ec2.internal 26m Normal Scheduled pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 Successfully assigned openshift-ingress/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 to ip-10-0-140-206.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct to ip-10-0-130-169.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-fpvjm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-fpvjm to ip-10-0-130-169.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-j9fwr Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-j9fwr to ip-10-0-130-169.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft to ip-10-0-140-206.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-r87zd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-r87zd to ip-10-0-140-206.ec2.internal 39m Normal Scheduled pod/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp Successfully assigned openshift-ingress/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l to ip-10-0-140-206.ec2.internal 42m Normal Scheduled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c Successfully assigned openshift-ingress/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c to ip-10-0-143-7.ec2.internal 22m Warning FailedScheduling pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-tcrmb Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-tcrmb to ip-10-0-140-206.ec2.internal 19m Warning FailedScheduling pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc Successfully assigned openshift-ingress/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc to ip-10-0-130-169.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs Successfully assigned openshift-ingress/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 Successfully assigned openshift-ingress/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 to ip-10-0-140-206.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-rmjfn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-rmjfn to ip-10-0-130-169.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph to ip-10-0-130-169.ec2.internal 40m Normal Scheduled pod/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj Successfully assigned openshift-ingress/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj to ip-10-0-143-7.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-4be164-5676d98555-tgtpr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-4be164-5676d98555-tgtpr to ip-10-0-140-206.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-q47wt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-q47wt to ip-10-0-130-169.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-b-7e233f-b5cfd5c8-8nwls Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-7e233f-b5cfd5c8-8nwls to ip-10-0-130-169.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-s86ld Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-s86ld to ip-10-0-143-7.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk to ip-10-0-140-206.ec2.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-b-e69dee-f496f5d46-npq8n Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-e69dee-f496f5d46-npq8n to ip-10-0-130-169.ec2.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-fbkj2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-fbkj2 to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s Successfully assigned openshift-ingress/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s to ip-10-0-130-169.ec2.internal 46m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p to ip-10-0-130-169.ec2.internal 26m Normal Scheduled pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz Successfully assigned openshift-ingress/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz to ip-10-0-140-206.ec2.internal 44m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg to ip-10-0-130-169.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-rht6s Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-rht6s to ip-10-0-130-169.ec2.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 to ip-10-0-130-169.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr to ip-10-0-130-169.ec2.internal 38m Normal Scheduled pod/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn Successfully assigned openshift-ingress/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn to ip-10-0-143-7.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w to ip-10-0-140-206.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 to ip-10-0-140-206.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg to ip-10-0-130-169.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf to ip-10-0-130-169.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc to ip-10-0-130-169.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 to ip-10-0-130-169.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht to ip-10-0-130-169.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps to ip-10-0-130-169.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp to ip-10-0-130-169.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 to ip-10-0-140-206.ec2.internal 37m Normal Scheduled pod/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm Successfully assigned openshift-ingress/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm to ip-10-0-130-169.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd to ip-10-0-140-206.ec2.internal 21m Warning FailedScheduling pod/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 50m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 to ip-10-0-130-169.ec2.internal 44m Normal Scheduled pod/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 Successfully assigned openshift-ingress/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 to ip-10-0-130-169.ec2.internal 28m Normal Scheduled pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Successfully assigned openshift-ingress/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb to ip-10-0-140-206.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj to ip-10-0-143-7.ec2.internal 22m Normal Scheduled pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp Successfully assigned openshift-ingress/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp to ip-10-0-130-169.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Successfully assigned openshift-ingress/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd to ip-10-0-143-7.ec2.internal 42m Normal Scheduled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk Successfully assigned openshift-ingress/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk to ip-10-0-143-7.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 to ip-10-0-143-7.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-a-50d078-58ffdcc78d-qh9nz Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-50d078-58ffdcc78d-qh9nz to ip-10-0-130-169.ec2.internal 46m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx to ip-10-0-143-7.ec2.internal 42m Normal Scheduled pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Successfully assigned openshift-ingress/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 to ip-10-0-130-169.ec2.internal 26m Normal Scheduled pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 Successfully assigned openshift-ingress/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 to ip-10-0-130-169.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp Successfully assigned openshift-ingress/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp to ip-10-0-140-206.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 to ip-10-0-143-7.ec2.internal 31m Normal Scheduled pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d Successfully assigned openshift-ingress/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d to ip-10-0-143-7.ec2.internal 36m Normal Scheduled pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Successfully assigned openshift-ingress/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-cg2gj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-cg2gj to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 Successfully assigned openshift-ingress/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 to ip-10-0-143-7.ec2.internal 45m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn to ip-10-0-140-206.ec2.internal 47m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd to ip-10-0-130-169.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-tqh66 Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-tqh66 to ip-10-0-130-169.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-api-b-da60de-dd5598cb7-fxnsc Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-da60de-dd5598cb7-fxnsc to ip-10-0-140-206.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-9xs79 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-9xs79 to ip-10-0-130-169.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-b-07410c-67d6df4667-nwwnp Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-07410c-67d6df4667-nwwnp to ip-10-0-130-169.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-api-a-260268-89664cb59-4p45z Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-260268-89664cb59-4p45z to ip-10-0-130-169.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-a-17d855-7579cb887-tl257 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-17d855-7579cb887-tl257 to ip-10-0-140-206.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-6bhcg Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-6bhcg to ip-10-0-140-206.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 to ip-10-0-130-169.ec2.internal 46m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb to ip-10-0-130-169.ec2.internal 45m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh to ip-10-0-143-7.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl to ip-10-0-140-206.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s to ip-10-0-143-7.ec2.internal 30m Normal Scheduled pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Successfully assigned openshift-ingress/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv to ip-10-0-140-206.ec2.internal 46m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl to ip-10-0-130-169.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m to ip-10-0-130-169.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-ait-b61ed672-65dd489d55-tfgts Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-b61ed672-65dd489d55-tfgts to ip-10-0-140-206.ec2.internal 36m Normal Scheduled pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Successfully assigned openshift-ingress/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj to ip-10-0-130-169.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 to ip-10-0-143-7.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 to ip-10-0-130-169.ec2.internal 43m Normal Scheduled pod/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 Successfully assigned openshift-ingress/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 to ip-10-0-143-7.ec2.internal 26m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 to ip-10-0-130-169.ec2.internal 34m Normal Scheduled pod/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j Successfully assigned openshift-ingress/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j to ip-10-0-140-206.ec2.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-7sck9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-7sck9 to ip-10-0-130-169.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-rrlcc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-rrlcc to ip-10-0-140-206.ec2.internal 46m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 to ip-10-0-143-7.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-gbhqt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-gbhqt to ip-10-0-130-169.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-vkxqt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-vkxqt to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j Successfully assigned openshift-ingress/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt Successfully assigned openshift-ingress/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt to ip-10-0-140-206.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb to ip-10-0-140-206.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 to ip-10-0-140-206.ec2.internal 23m Normal Scheduled pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Successfully assigned openshift-ingress/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w to ip-10-0-130-169.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-7b98bb678c-txfcv Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-txfcv to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Successfully assigned openshift-ingress/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj to ip-10-0-130-169.ec2.internal 19m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 36m Normal Scheduled pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Successfully assigned openshift-ingress/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 to ip-10-0-143-7.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp to ip-10-0-143-7.ec2.internal 33m Normal Scheduled pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh Successfully assigned openshift-ingress/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh to ip-10-0-130-169.ec2.internal 48m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq to ip-10-0-140-206.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r to ip-10-0-130-169.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-api-b-288da6-b54cbc654-wx2zn Successfully assigned openshift-ingress/payload-processing-e2e-api-b-288da6-b54cbc654-wx2zn to ip-10-0-140-206.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-api-b-07410c-7598fb6b98-8r4jm Successfully assigned openshift-ingress/payload-processing-e2e-api-b-07410c-7598fb6b98-8r4jm to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 to ip-10-0-130-169.ec2.internal 18m Normal Scheduled pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Successfully assigned openshift-ingress/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw to ip-10-0-140-206.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w Successfully assigned openshift-ingress/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w to ip-10-0-143-7.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s to ip-10-0-130-169.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db to ip-10-0-140-206.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl to ip-10-0-140-206.ec2.internal 50m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Successfully assigned openshift-ingress/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp to ip-10-0-130-169.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm to ip-10-0-140-206.ec2.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-a-29bb56-b9f5594c8-kms8b Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-29bb56-b9f5594c8-kms8b to ip-10-0-130-169.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-a-368bbb-55769f5bb4-8bhjd Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-368bbb-55769f5bb4-8bhjd to ip-10-0-143-7.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-49xsh Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-49xsh to ip-10-0-130-169.ec2.internal 19m Normal Scheduled pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 Successfully assigned openshift-ingress/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 to ip-10-0-143-7.ec2.internal 19m Warning FailedScheduling pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Warning FailedScheduling pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 46m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 to ip-10-0-143-7.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 to ip-10-0-130-169.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq to ip-10-0-140-206.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Successfully assigned openshift-ingress/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb to ip-10-0-130-169.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Successfully assigned openshift-ingress/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps to ip-10-0-143-7.ec2.internal 30m Normal Scheduled pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Successfully assigned openshift-ingress/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk to ip-10-0-130-169.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 to ip-10-0-143-7.ec2.internal 21m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-6c62j 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 26m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-mrztt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-mrztt to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-4be164-646f4df6bb-8bxsj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-4be164-646f4df6bb-8bxsj to ip-10-0-143-7.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-scn6w to ip-10-0-140-206.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 to ip-10-0-140-206.ec2.internal 19m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-6c62j 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-cttw4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-cttw4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 35m Normal Scheduled pod/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr Successfully assigned openshift-ingress/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr to ip-10-0-140-206.ec2.internal 24m Normal Scheduled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Successfully assigned openshift-ingress/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 to ip-10-0-130-169.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-api-a-260268-5558bb6674-bwd6f Successfully assigned openshift-ingress/payload-processing-e2e-api-a-260268-5558bb6674-bwd6f to ip-10-0-130-169.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-pwmfm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-pwmfm to ip-10-0-140-206.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-4be164-6d67f7db44-ll6wk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-4be164-6d67f7db44-ll6wk to ip-10-0-130-169.ec2.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-pc85x to ip-10-0-130-169.ec2.internal 46m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 to ip-10-0-140-206.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t to ip-10-0-130-169.ec2.internal 19m Normal Scheduled pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Successfully assigned openshift-ingress/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz to ip-10-0-130-169.ec2.internal 19m Warning FailedScheduling pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Normal Scheduled pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 Successfully assigned openshift-ingress/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 to ip-10-0-130-169.ec2.internal 41m Normal Scheduled pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Successfully assigned openshift-ingress/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj to ip-10-0-130-169.ec2.internal 21m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 70m Warning FailedScheduling pod/router-default-75647bc475-s8spf no nodes available to schedule pods 51m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lbh6w to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj to ip-10-0-130-169.ec2.internal 20m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 64m Warning FailedScheduling pod/router-default-75647bc475-s8spf no nodes available to schedule pods 40m Normal Scheduled pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Successfully assigned openshift-ingress/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 to ip-10-0-130-169.ec2.internal 64m Normal Scheduled pod/router-default-75647bc475-s8spf Successfully assigned openshift-ingress/router-default-75647bc475-s8spf to ip-10-0-130-169.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq Successfully assigned openshift-ingress/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq to ip-10-0-130-169.ec2.internal 21m Normal Scheduled pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d Successfully assigned openshift-ingress/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d to ip-10-0-140-206.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-j726m Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j726m to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-73831d-75bf4f98d-2lkh5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-73831d-75bf4f98d-2lkh5 to ip-10-0-130-169.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn to ip-10-0-130-169.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-fwnrr to ip-10-0-130-169.ec2.internal 21m Warning FailedScheduling pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Warning FailedScheduling pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/payload-processing-e2e-api-a-17d855-7cd78798fd-tdv5f Successfully assigned openshift-ingress/payload-processing-e2e-api-a-17d855-7cd78798fd-tdv5f to ip-10-0-140-206.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 to ip-10-0-140-206.ec2.internal 48m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 to ip-10-0-140-206.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp to ip-10-0-143-7.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-fj9f6 to ip-10-0-130-169.ec2.internal 19m Normal Scheduled pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Successfully assigned openshift-ingress/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w to ip-10-0-143-7.ec2.internal 19m Warning FailedScheduling pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 20m Warning FailedScheduling pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 46m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 to ip-10-0-140-206.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq to ip-10-0-140-206.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 to ip-10-0-140-206.ec2.internal 58m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-6f89d9f6ccqmw9g Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6f89d9f6ccqmw9g to ip-10-0-143-7.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-f6dvm to ip-10-0-143-7.ec2.internal 22m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Warning FailedScheduling pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb to ip-10-0-143-7.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt to ip-10-0-130-169.ec2.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-f5pvt to ip-10-0-143-7.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 to ip-10-0-130-169.ec2.internal 32m Normal Scheduled pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Successfully assigned openshift-ingress/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk to ip-10-0-130-169.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-wzfkc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-wzfkc to ip-10-0-130-169.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-dzvm2 to ip-10-0-140-206.ec2.internal 46m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv to ip-10-0-143-7.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-k7kp5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-k7kp5 to ip-10-0-130-169.ec2.internal 46m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm to ip-10-0-140-206.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v to ip-10-0-140-206.ec2.internal 46m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk to ip-10-0-130-169.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt to ip-10-0-143-7.ec2.internal 52m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-bq26k to ip-10-0-130-169.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs to ip-10-0-143-7.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-b88g4 to ip-10-0-143-7.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj to ip-10-0-130-169.ec2.internal 46m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg to ip-10-0-143-7.ec2.internal 46m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd to ip-10-0-143-7.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl to ip-10-0-143-7.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx to ip-10-0-140-206.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 to ip-10-0-140-206.ec2.internal 46m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg to ip-10-0-130-169.ec2.internal 29m Normal Scheduled pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Successfully assigned openshift-ingress/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 to ip-10-0-143-7.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-9x6g8 to ip-10-0-140-206.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-dbpp4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-dbpp4 to ip-10-0-130-169.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h to ip-10-0-130-169.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-92m4m to ip-10-0-130-169.ec2.internal 22m Warning FailedScheduling pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp to ip-10-0-130-169.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 Successfully assigned openshift-ingress/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 to ip-10-0-140-206.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq to ip-10-0-130-169.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz Successfully assigned openshift-ingress/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz to ip-10-0-130-169.ec2.internal 47m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj to ip-10-0-140-206.ec2.internal 29m Normal Scheduled pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Successfully assigned openshift-ingress/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d to ip-10-0-143-7.ec2.internal 42m Normal Scheduled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Successfully assigned openshift-ingress/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh to ip-10-0-130-169.ec2.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-8ntx4 to ip-10-0-143-7.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-679bd2-55ff675f5d-zjh5p Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-679bd2-55ff675f5d-zjh5p to ip-10-0-130-169.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Successfully assigned openshift-ingress/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc to ip-10-0-143-7.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-13a3e2-d4978667b-txkrw Successfully assigned openshift-ingress/payload-processing-e2e-api-a-13a3e2-d4978667b-txkrw to ip-10-0-140-206.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj Successfully assigned openshift-ingress/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj to ip-10-0-130-169.ec2.internal 27m Normal Scheduled pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw Successfully assigned openshift-ingress/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw to ip-10-0-140-206.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-ait-b61ed672-65b7c7796d-5g7zk Successfully assigned openshift-ingress/payload-processing-e2e-ait-b61ed672-65b7c7796d-5g7zk to ip-10-0-130-169.ec2.internal 21m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-rlp5s 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Normal Scheduled pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 Successfully assigned openshift-ingress/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 to ip-10-0-140-206.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-7r8hn to ip-10-0-143-7.ec2.internal 19m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-rlp5s 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 48m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd to ip-10-0-143-7.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss Successfully assigned openshift-ingress/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss to ip-10-0-143-7.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r to ip-10-0-130-169.ec2.internal 46m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 to ip-10-0-140-206.ec2.internal 26m Normal Scheduled pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 Successfully assigned openshift-ingress/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 to ip-10-0-130-169.ec2.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-4zkk6 to ip-10-0-140-206.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 to ip-10-0-130-169.ec2.internal 21m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wzwfm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wzwfm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 30m Normal Scheduled pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Successfully assigned openshift-ingress/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf to ip-10-0-130-169.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z Successfully assigned openshift-ingress/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z to ip-10-0-130-169.ec2.internal 19m Normal Scheduled pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Successfully assigned openshift-ingress/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 to ip-10-0-140-206.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-ait-0660f91f-c4956954b-s2lkw Successfully assigned openshift-ingress/payload-processing-e2e-ait-0660f91f-c4956954b-s2lkw to ip-10-0-130-169.ec2.internal 19m Warning FailedScheduling pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 26m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-9z2kk Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-9z2kk to ip-10-0-130-169.ec2.internal 22m Warning FailedScheduling pod/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 34m Normal Scheduled pod/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 Successfully assigned openshift-ingress/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 to ip-10-0-130-169.ec2.internal 22m Warning FailedScheduling pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Warning FailedScheduling pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Normal Scheduled pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr Successfully assigned openshift-ingress/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr to ip-10-0-130-169.ec2.internal 58m Normal Scheduled pod/kube-auth-proxy-6cfd9d8567-zfw8x Successfully assigned openshift-ingress/kube-auth-proxy-6cfd9d8567-zfw8x to ip-10-0-140-206.ec2.internal 48m Normal Scheduled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Successfully assigned openshift-ingress/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 to ip-10-0-140-206.ec2.internal 58m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-zhsns Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-zhsns to ip-10-0-130-169.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l Successfully assigned openshift-ingress/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l to ip-10-0-143-7.ec2.internal 57m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-6bdqg Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6bdqg to ip-10-0-130-169.ec2.internal 38m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-n6c2h Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n6c2h to ip-10-0-130-169.ec2.internal 22m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-ppsvj to ip-10-0-140-206.ec2.internal 58m Normal Scheduled pod/kube-auth-proxy-6cfd9d8567-cqfzb Successfully assigned openshift-ingress/kube-auth-proxy-6cfd9d8567-cqfzb to ip-10-0-130-169.ec2.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-679bd2-658f485b5b-fbjjm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-679bd2-658f485b5b-fbjjm to ip-10-0-130-169.ec2.internal 21m Warning FailedScheduling pod/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 20m Warning FailedScheduling pod/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 35m Normal Scheduled pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Successfully assigned openshift-ingress/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 to ip-10-0-130-169.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw to ip-10-0-143-7.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh to ip-10-0-140-206.ec2.internal 8m54s Normal Scheduled pod/payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b to ip-10-0-140-206.ec2.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v5jpm to ip-10-0-140-206.ec2.internal 19m Warning FailedScheduling pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 19m Warning FailedScheduling pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 32m Normal Scheduled pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 Successfully assigned openshift-ingress/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 to ip-10-0-143-7.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s to ip-10-0-143-7.ec2.internal 19m Normal Scheduled pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Successfully assigned openshift-ingress/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr to ip-10-0-143-7.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz to ip-10-0-130-169.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx to ip-10-0-140-206.ec2.internal 46m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 to ip-10-0-130-169.ec2.internal 43m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-xq4mm Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xq4mm to ip-10-0-143-7.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw to ip-10-0-140-206.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk to ip-10-0-130-169.ec2.internal 45m Normal Scheduled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Successfully assigned openshift-ingress/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr to ip-10-0-130-169.ec2.internal 35m Normal Scheduled pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh Successfully assigned openshift-ingress/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh to ip-10-0-130-169.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq Successfully assigned openshift-ingress/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq to ip-10-0-140-206.ec2.internal 5m26s Normal Scheduled pod/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd to ip-10-0-140-206.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp to ip-10-0-130-169.ec2.internal 71m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 71m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 71m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 70m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-75647bc475 from 0 to 1 70m Normal SuccessfulCreate replicaset/router-default-75647bc475 Created pod: router-default-75647bc475-s8spf 64m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 64m Warning FailedMount pod/router-default-75647bc475-s8spf MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 64m Warning FailedMount pod/router-default-75647bc475-s8spf MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 63m Warning FailedMount pod/router-default-75647bc475-s8spf MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 63m Warning FailedMount pod/router-default-75647bc475-s8spf MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 63m Normal AddedInterface pod/router-default-75647bc475-s8spf Add eth0 [10.132.0.13/23] from ovn-kubernetes 63m Normal Started pod/router-default-75647bc475-s8spf Container started 63m Normal Created pod/router-default-75647bc475-s8spf Container created 63m Normal Pulled pod/router-default-75647bc475-s8spf Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/kube-auth-proxy-6cfd9d8567 Created pod: kube-auth-proxy-6cfd9d8567-zfw8x 58m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-6cfd9d8567 from 0 to 2 58m Normal SuccessfulCreate replicaset/kube-auth-proxy-6cfd9d8567 Created pod: kube-auth-proxy-6cfd9d8567-cqfzb 58m Warning FailedMount pod/kube-auth-proxy-6cfd9d8567-cqfzb MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found 58m Warning FailedMount pod/kube-auth-proxy-6cfd9d8567-zfw8x MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found 58m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-zhsns 58m Normal AddedInterface pod/kube-auth-proxy-6cfd9d8567-cqfzb Add eth0 [10.132.0.33/23] from ovn-kubernetes 58m Normal AddedInterface pod/kube-auth-proxy-6cfd9d8567-zfw8x Add eth0 [10.133.0.20/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 58m Normal Pulling pod/kube-auth-proxy-6cfd9d8567-zfw8x Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 58m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-zhsns Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 58m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-zhsns Add eth0 [10.132.0.34/23] from ovn-kubernetes 58m Normal Pulling pod/kube-auth-proxy-6cfd9d8567-cqfzb Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 58m Normal Created pod/kube-auth-proxy-6cfd9d8567-zfw8x Container created 58m Normal Created pod/kube-auth-proxy-6cfd9d8567-cqfzb Container created 58m Normal Started pod/kube-auth-proxy-6cfd9d8567-cqfzb Container started 58m Normal Pulled pod/kube-auth-proxy-6cfd9d8567-cqfzb Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.634s (2.634s including waiting). Image size: 188904677 bytes. 58m Normal Started pod/kube-auth-proxy-6cfd9d8567-zfw8x Container started 58m Normal Pulled pod/kube-auth-proxy-6cfd9d8567-zfw8x Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.235s (2.235s including waiting). Image size: 188904677 bytes. 58m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-zhsns Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.537s (2.537s including waiting). Image size: 185476477 bytes. 58m Normal Started pod/istiod-openshift-gateway-7876bfcb74-zhsns Container started 58m Normal Created pod/istiod-openshift-gateway-7876bfcb74-zhsns Container created 58m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-6f89d9f6ccqmw9g Add eth0 [10.134.0.26/23] from ovn-kubernetes 58m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-zhsns Readiness probe error: HTTP probe failed with statuscode: 503... 58m Normal Pulling pod/data-science-gateway-data-science-gateway-class-6f89d9f6ccqmw9g Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 58m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-6f89d9f6cc from 0 to 1 58m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-zhsns Readiness probe failed: HTTP probe failed with statuscode: 503 58m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-6f89d9f6cc Created pod: data-science-gateway-data-science-gateway-class-6f89d9f6ccqmw9g 58m Normal Pulled pod/data-science-gateway-data-science-gateway-class-6f89d9f6ccqmw9g Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.759s (2.759s including waiting). Image size: 183555333 bytes. 58m Normal Created pod/data-science-gateway-data-science-gateway-class-6f89d9f6ccqmw9g Container created 58m Normal Started pod/data-science-gateway-data-science-gateway-class-6f89d9f6ccqmw9g Container started 58m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 57m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-6bdqg Container started 57m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-6bdqg Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-6bdqg Add eth0 [10.132.0.36/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-6bdqg 57m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-6bdqg Container created 57m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-6bdqg Readiness probe error: HTTP probe failed with statuscode: 503... 57m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-6bdqg Readiness probe failed: HTTP probe failed with statuscode: 503 57m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 57m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-zhsns 57m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-zhsns Stopping container discovery 57m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-8ntx4 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-67bf986f99 from 0 to 1 55m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Add eth0 [10.134.0.37/23] from ovn-kubernetes 53m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 53m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Container created 53m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Container started 53m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 53m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-67bf986f99 from 1 to 2 52m Normal Pulling pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 52m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Add eth0 [10.132.0.45/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-bq26k 52m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Container started 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.393s (2.393s including waiting). Image size: 183555333 bytes. 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Container created 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused... 51m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-nkqms 51m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 51m Normal AddedInterface pod/payload-processing-5f78687d5b-nkqms Add eth0 [10.133.0.41/23] from ovn-kubernetes 51m Normal Pulling pod/payload-processing-5f78687d5b-nkqms Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 51m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-txfcv 51m Normal Pulling pod/payload-pre-processing-7b98bb678c-txfcv Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 51m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-txfcv Add eth0 [10.132.0.47/23] from ovn-kubernetes 51m Normal Created pod/payload-processing-5f78687d5b-nkqms Container created 51m Normal Pulled pod/payload-pre-processing-7b98bb678c-txfcv Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.216s (2.216s including waiting). Image size: 125317101 bytes. 51m Normal Pulled pod/payload-processing-5f78687d5b-nkqms Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.89s (1.89s including waiting). Image size: 125317101 bytes. 51m Normal Started pod/payload-processing-5f78687d5b-nkqms Container started 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-fj9f6 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-9x6g8 51m Normal Pulling pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 51m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-67bf986f99 from 2 to 4 51m Normal Created pod/payload-pre-processing-7b98bb678c-txfcv Container created 51m Normal Started pod/payload-pre-processing-7b98bb678c-txfcv Container started 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Add eth0 [10.133.0.42/23] from ovn-kubernetes 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Add eth0 [10.132.0.48/23] from ovn-kubernetes 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Container created 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Container started 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Container created 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.259s (3.259s including waiting). Image size: 183555333 bytes. 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Container started 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-pwmfm Add eth0 [10.133.0.44/23] from ovn-kubernetes 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Add eth0 [10.134.0.39/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-pwmfm 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-f6dvm 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-j726m 51m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-j726m Add eth0 [10.132.0.50/23] from ovn-kubernetes 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Add eth0 [10.133.0.43/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-4zkk6 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-j726m Container started 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Container started 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Container created 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-pwmfm Container started 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-j726m Container created 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-j726m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Container started 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-pwmfm Container created 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Container created 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-pwmfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-pwmfm Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-j726m Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused... 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-pwmfm Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused... 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-j726m Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Add eth0 [10.134.0.40/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 (combined from similar events): Created pod: maas-default-gateway-openshift-default-67bf986f99-lbh6w 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-7r8hn 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Add eth0 [10.132.0.51/23] from ovn-kubernetes 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Container created 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Container created 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Container started 51m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Container started 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused... 51m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 50m Normal AddedLabel gateway/e2e-worker-w1-c104b8 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-c104b8 50m Normal AddedLabel gateway/e2e-worker-w2-fd3281 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-fd3281 50m Normal AddedLabel gateway/e2e-worker-w5-dca38d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-dca38d 50m Normal AddedLabel gateway/e2e-worker-w6-c4847c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-c4847c 50m Normal AddedLabel gateway/e2e-worker-w3-82b647 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-82b647 50m Normal AddedLabel gateway/e2e-worker-w0-ba3d7b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-ba3d7b 50m Normal AddedLabel gateway/e2e-worker-w4-d827b7 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-d827b7 50m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled up replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 0 to 1 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled up replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 0 to 1 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-c4847c-openshift-default Scaled up replica set e2e-worker-w6-c4847c-openshift-default-65bf677b7f from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Created pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled up replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm 50m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf Add eth0 [10.132.0.52/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled up replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba3d7b-openshift-default Scaled up replica set e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-82b647-openshift-default Scaled up replica set e2e-worker-w3-82b647-openshift-default-6747d4d57c from 0 to 1 50m Normal AddedInterface pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt Add eth0 [10.133.0.46/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-82b647-openshift-default-6747d4d57c Created pod: e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt 50m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Add eth0 [10.133.0.45/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k Add eth0 [10.133.0.47/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Add eth0 [10.132.0.53/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Add eth0 [10.132.0.54/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z Add eth0 [10.132.0.55/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-dca38d-745fb76666 Created pod: payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-dca38d Scaled up replica set payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555 Created pod: payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555 Error creating: pods "payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-dca38d: serviceaccount "payload-processing-e2e-worker-w5-dca38d" not found 50m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-dca38d-745fb76666 Error creating: pods "payload-processing-e2e-worker-w5-dca38d-745fb76666-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-dca38d: serviceaccount "payload-processing-e2e-worker-w5-dca38d" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-dca38d Scaled up replica set payload-processing-e2e-worker-w5-dca38d-745fb76666 from 0 to 1 50m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm Add eth0 [10.133.0.49/23] from ovn-kubernetes 50m Normal Started pod/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm Container started 50m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb Add eth0 [10.133.0.48/23] from ovn-kubernetes 50m Normal Created pod/payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb Container created 50m Normal Pulled pod/payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb Container started 50m Normal Created pod/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm Container created 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-c104b8-6c778f644d Created pod: payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5 Created pod: payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-c104b8 Scaled up replica set payload-processing-e2e-worker-w1-c104b8-6c778f644d from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-c104b8-6c778f644d Error creating: pods "payload-processing-e2e-worker-w1-c104b8-6c778f644d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-c104b8: serviceaccount "payload-processing-e2e-worker-w1-c104b8" not found 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5 Error creating: pods "payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-c104b8: serviceaccount "payload-processing-e2e-worker-w1-c104b8" not found 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-c104b8 Scaled up replica set payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95 Created pod: payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc 50m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-ba3d7b Scaled up replica set payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95 from 0 to 1 50m Normal Started pod/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 Container started 50m Normal AddedInterface pod/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 Add eth0 [10.132.0.56/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-ba3d7b Scaled up replica set payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88 from 0 to 1 50m Normal Pulled pod/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 Container created 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95 Error creating: pods "payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ba3d7b: serviceaccount "payload-processing-e2e-worker-w0-ba3d7b" not found 50m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 Add eth0 [10.132.0.57/23] from ovn-kubernetes 50m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88 Error creating: pods "payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ba3d7b: serviceaccount "payload-processing-e2e-worker-w0-ba3d7b" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88 Created pod: payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp 50m Normal Started pod/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 Container started 50m Normal Created pod/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 Container created 50m Normal Created pod/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc Container created 50m Normal AddedInterface pod/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp Add eth0 [10.132.0.58/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp Container started 50m Normal Created pod/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp Container created 50m Normal Pulled pod/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-82b647 Scaled up replica set payload-pre-processing-e2e-worker-w3-82b647-546f5847f from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-82b647-7b45bddbb6 Error creating: pods "payload-processing-e2e-worker-w3-82b647-7b45bddbb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-82b647: serviceaccount "payload-processing-e2e-worker-w3-82b647" not found 50m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc Add eth0 [10.132.0.59/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-82b647 Scaled up replica set payload-processing-e2e-worker-w3-82b647-7b45bddbb6 from 0 to 1 50m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-82b647-546f5847f Error creating: pods "payload-pre-processing-e2e-worker-w3-82b647-546f5847f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-82b647: serviceaccount "payload-processing-e2e-worker-w3-82b647" not found 50m Normal Started pod/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc Container started 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f Created pod: payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-fd3281 Scaled up replica set payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-82b647-546f5847f Created pod: payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r 50m Normal Created pod/payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r Container created 50m Normal Started pod/payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s Container started 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-fd3281-864d489c66 Created pod: payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl 50m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r Container started 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-fd3281 Scaled up replica set payload-processing-e2e-worker-w2-fd3281-864d489c66 from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-fd3281-864d489c66 Error creating: pods "payload-processing-e2e-worker-w2-fd3281-864d489c66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-fd3281: serviceaccount "payload-processing-e2e-worker-w2-fd3281" not found 50m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r Add eth0 [10.132.0.60/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s Add eth0 [10.132.0.61/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s Container created 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f Error creating: pods "payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-fd3281: serviceaccount "payload-processing-e2e-worker-w2-fd3281" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-82b647-7b45bddbb6 Created pod: payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s 50m Normal Pulled pod/payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl Add eth0 [10.133.0.53/23] from ovn-kubernetes 50m Normal Started pod/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw Container started 50m Normal Created pod/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw Container created 50m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw Add eth0 [10.133.0.52/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl Container started 50m Normal Created pod/payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl Container created 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c Created pod: payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r 50m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r Add eth0 [10.132.0.62/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 Add eth0 [10.133.0.54/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-c4847c Scaled up replica set payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56 Created pod: payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c Error creating: pods "payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-c4847c: serviceaccount "payload-processing-e2e-worker-w6-c4847c" not found 50m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56 Error creating: pods "payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-c4847c: serviceaccount "payload-processing-e2e-worker-w6-c4847c" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-c4847c Scaled up replica set payload-processing-e2e-worker-w6-c4847c-65f8bb6c56 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-d827b7 Scaled up replica set payload-processing-e2e-worker-w4-d827b7-6d9bd77954 from 0 to 1 50m Normal Started pod/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 Container started 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-d827b7-6d9bd77954 Created pod: payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff Error creating: pods "payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-d827b7: serviceaccount "payload-processing-e2e-worker-w4-d827b7" not found 50m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 Add eth0 [10.132.0.64/23] from ovn-kubernetes 50m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-d827b7-6d9bd77954 Error creating: pods "payload-processing-e2e-worker-w4-d827b7-6d9bd77954-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-d827b7: serviceaccount "payload-processing-e2e-worker-w4-d827b7" not found 50m Normal Started pod/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 Container started 50m Normal AddedInterface pod/payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj Add eth0 [10.132.0.63/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff Created pod: payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 50m Normal Started pod/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r Container started 50m Normal Pulled pod/payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 Container created 50m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-d827b7 Scaled up replica set payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff from 0 to 1 50m Normal Created pod/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r Container created 50m Normal Created pod/payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj Container created 50m Normal Started pod/payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj Container started 50m Normal Created pod/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 Container created 50m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Container created 50m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Container started 50m Normal Pulled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k Container created 50m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused... 50m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Container started 50m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Container created 50m Normal Started pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k Container started 50m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning Unhealthy pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused... 50m Normal Pulled pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt Container started 50m Normal Created pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt Container created 50m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Container started 50m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Container created 50m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z Container started 50m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z Container created 50m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf Container created 50m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf Container started 50m Warning ProbeError pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-82b647-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-82b647-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-82b647-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-82b647-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Add eth0 [10.134.0.49/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled up replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 1 to 2 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled up replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 1 to 2 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl 49m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Add eth0 [10.132.0.71/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Container started 49m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Container created 49m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Container started 49m Normal Created pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Container created 49m Normal Pulled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba3d7b-openshift-default Scaled up replica set e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 from 1 to 2 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Add eth0 [10.134.0.50/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Container started 49m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Container created 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Created pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 48m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-82b647-openshift-default-6747d4d57c Created pod: e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-82b647-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-c4847c-openshift-default Scaled up replica set e2e-worker-w6-c4847c-openshift-default-65bf677b7f from 1 to 2 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-82b647-openshift-default Scaled up replica set e2e-worker-w3-82b647-openshift-default-6747d4d57c from 1 to 2 48m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Container created 48m Normal AddedInterface pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Add eth0 [10.134.0.51/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Add eth0 [10.134.0.52/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd 48m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Container started 48m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Add eth0 [10.133.0.61/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled up replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 1 to 2 48m Normal Started pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Container started 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal Created pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Container created 48m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Container created 48m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Container started 48m Warning ProbeError pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled up replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 1 to 2 48m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Add eth0 [10.133.0.62/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Container started 48m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Container created 48m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 48m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled up replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 2 to 3 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 48m Normal Pulled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Container created 48m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Container started 48m Normal AddedInterface pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Add eth0 [10.134.0.53/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Container started 48m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Container created 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Created pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf 48m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba3d7b-openshift-default Scaled up replica set e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 from 2 to 3 48m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Add eth0 [10.133.0.63/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 48m Normal Pulled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba3d7b-openshift-default Scaled up replica set e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 from 3 to 4 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal AddedInterface pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l Add eth0 [10.132.0.86/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled up replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 2 to 3 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Created pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l 48m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Add eth0 [10.133.0.65/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l Container created 48m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Container created 48m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Container started 48m Normal Started pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l Container started 47m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Add eth0 [10.134.0.55/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Add eth0 [10.132.0.88/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 47m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Container created 47m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Container started 47m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Container created 47m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd 47m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Container started 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-c4847c-openshift-default Scaled up replica set e2e-worker-w6-c4847c-openshift-default-65bf677b7f from 2 to 4 47m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Add eth0 [10.134.0.56/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled up replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 2 to 3 47m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Container created 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 47m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Container started 47m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 47m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled up replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 3 to 4 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj 47m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Container started 47m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Container created 47m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Add eth0 [10.133.0.66/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 47m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled up replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 2 to 3 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd 47m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Container created 47m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Container started 47m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Add eth0 [10.132.0.91/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd 47m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Add eth0 [10.134.0.58/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq 47m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Add eth0 [10.132.0.95/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-c4847c-openshift-default Scaled up replica set e2e-worker-w6-c4847c-openshift-default-65bf677b7f from 4 to 8 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 47m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 47m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Add eth0 [10.133.0.68/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Add eth0 [10.133.0.70/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Add eth0 [10.134.0.59/23] from ovn-kubernetes 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 47m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Add eth0 [10.134.0.57/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Container created 47m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled up replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 3 to 6 47m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Container created 47m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Container started 47m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Container started 47m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Container started 47m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Add eth0 [10.132.0.94/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Container created 47m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Container started 47m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Container created 47m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Container created 47m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Container started 47m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Container started 47m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Container created 47m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Container started 47m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Container created 46m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd 46m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Add eth0 [10.132.0.97/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p 46m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx 46m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 Add eth0 [10.133.0.71/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 46m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Add eth0 [10.134.0.60/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv 46m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 46m Normal SuccessfulCreate replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c (combined from similar events): Created pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 46m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled up replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 4 to 8 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 10; reason: 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled up replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 6 to 10 46m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Container created 46m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Container started 46m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd Add eth0 [10.134.0.62/23] from ovn-kubernetes 46m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 Container created 46m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Container created 46m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd Container created 46m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd Container started 46m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 Container started 46m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Add eth0 [10.134.0.61/23] from ovn-kubernetes 46m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Container started 46m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Container created 46m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Add eth0 [10.132.0.98/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 Container started 46m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 Container created 46m Normal Started pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Container started 46m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 Add eth0 [10.133.0.72/23] from ovn-kubernetes 46m Normal Created pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Container created 46m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Container started 46m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Add eth0 [10.132.0.99/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Container created 46m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Container started 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 46m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 46m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled up replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 3 to 6 46m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Container created 46m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Container started 46m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Container started 46m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 Add eth0 [10.133.0.74/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Container created 46m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 Container started 46m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Add eth0 [10.132.0.107/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Add eth0 [10.134.0.66/23] from ovn-kubernetes 46m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 Container created 46m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Readiness probe error: Get "http://10.134.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Readiness probe failed: Get "http://10.134.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Readiness probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Readiness probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 46m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled up replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 6 to 7 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 7; reason: 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f (combined from similar events): Created pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 46m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 Container created 46m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 Container started 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-c4847c-openshift-default Scaled up replica set e2e-worker-w6-c4847c-openshift-default-65bf677b7f from 8 to 10 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 46m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 Add eth0 [10.132.0.109/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Add eth0 [10.134.0.67/23] from ovn-kubernetes 46m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Container created 46m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Container started 46m Normal AddedInterface pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Add eth0 [10.133.0.75/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Container created 46m Normal Started pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Container started 46m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 (combined from similar events): Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg 46m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled up replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 8 to 10 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Created pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg 46m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Add eth0 [10.132.0.115/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Add eth0 [10.134.0.68/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Container created 46m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Container started 45m Normal Pulled pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Container started 45m Normal Created pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Container created 45m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-v5jpm 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Add eth0 [10.133.0.76/23] from ovn-kubernetes 45m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Container created 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Container started 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.55s (2.55s including waiting). Image size: 185476477 bytes. 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 (combined from similar events): Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn 45m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled up replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 7 to 10 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Created pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh 45m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Readiness probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Readiness probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Readiness probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Container started 45m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Container started 45m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Container created 45m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Add eth0 [10.133.0.77/23] from ovn-kubernetes 45m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Container created 45m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Add eth0 [10.134.0.69/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Container created 45m Normal Started pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Container started 45m Normal AddedInterface pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Add eth0 [10.132.0.123/23] from ovn-kubernetes 45m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Startup probe failed: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Startup probe error: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Readiness probe error: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Readiness probe failed: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Readiness probe failed: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Readiness probe error: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Readiness probe failed: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Readiness probe error: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Readiness probe failed: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Readiness probe error: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Readiness probe error: Get "http://10.132.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Readiness probe failed: Get "http://10.132.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Readiness probe error: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Readiness probe error: Get "http://10.132.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Readiness probe failed: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Readiness probe error: Get "http://10.132.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Readiness probe failed: Get "http://10.132.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Readiness probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Normal AddedLabel gateway/e2e-ait-0660f91f Added label istio.io/rev=openshift-gateway to gateway e2e-ait-0660f91f 44m Normal ScalingReplicaSet deployment/e2e-ait-0660f91f-openshift-default Scaled up replica set e2e-ait-0660f91f-openshift-default-5b87f4869c from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-ait-0660f91f-openshift-default-5b87f4869c Created pod: e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz 44m Normal AddedInterface pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Add eth0 [10.132.0.135/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws Add eth0 [10.134.0.73/23] from ovn-kubernetes 44m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws Container started 44m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws Container created 44m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedLabel gateway/e2e-derive-8b58192f Added label istio.io/rev=openshift-gateway to gateway e2e-derive-8b58192f 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled up replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 3 to 4 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 44m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-0660f91f Scaled up replica set payload-processing-e2e-ait-0660f91f-c4956954b from 0 to 1 44m Normal ScalingReplicaSet deployment/e2e-derive-8b58192f-openshift-default Scaled up replica set e2e-derive-8b58192f-openshift-default-754c75cb97 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-derive-8b58192f-openshift-default-754c75cb97 Created pod: e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 44m Normal Created pod/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 Container created 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-0660f91f-c4956954b Created pod: payload-processing-e2e-ait-0660f91f-c4956954b-s2lkw 44m Normal AddedInterface pod/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 Add eth0 [10.132.0.137/23] from ovn-kubernetes 44m Warning FailedCreate replicaset/payload-processing-e2e-ait-0660f91f-c4956954b Error creating: pods "payload-processing-e2e-ait-0660f91f-c4956954b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0660f91f: serviceaccount "payload-processing-e2e-ait-0660f91f" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c Created pod: payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-gbhqt 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-0660f91f Scaled up replica set payload-pre-processing-e2e-ait-0660f91f-6f55c5558c from 0 to 1 44m Normal Pulled pod/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c Error creating: pods "payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0660f91f: serviceaccount "payload-processing-e2e-ait-0660f91f" not found 44m Normal Started pod/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 Container started 44m Normal Created pod/payload-processing-e2e-ait-0660f91f-c4956954b-s2lkw Container created 44m Normal AddedInterface pod/payload-processing-e2e-ait-0660f91f-c4956954b-s2lkw Add eth0 [10.132.0.139/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-ait-0660f91f-c4956954b-s2lkw Container started 44m Normal Started pod/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-gbhqt Container started 44m Normal Pulled pod/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-gbhqt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-gbhqt Add eth0 [10.132.0.138/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-ait-0660f91f-c4956954b-s2lkw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-gbhqt Container created 44m Normal AddedLabel gateway/e2e-ait-b61ed672 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-b61ed672 44m Normal ScalingReplicaSet deployment/e2e-ait-b61ed672-openshift-default Scaled up replica set e2e-ait-b61ed672-openshift-default-6cb675d767 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-ait-b61ed672-openshift-default-6cb675d767 Created pod: e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl 44m Normal AddedInterface pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Add eth0 [10.134.0.74/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-8b58192f Scaled up replica set payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-8b58192f Scaled up replica set payload-processing-e2e-derive-8b58192f-f48df6874 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-derive-8b58192f-f48df6874 Error creating: pods "payload-processing-e2e-derive-8b58192f-f48df6874-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-8b58192f: serviceaccount "payload-processing-e2e-derive-8b58192f" not found 44m Normal Pulled pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9 Error creating: pods "payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-8b58192f: serviceaccount "payload-processing-e2e-derive-8b58192f" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9 Created pod: payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-tqh66 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-8b58192f-f48df6874 Created pod: payload-processing-e2e-derive-8b58192f-f48df6874-wbtnf 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Readiness probe error: Get "http://10.133.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Normal Created pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Container created 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Readiness probe failed: Get "http://10.133.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal Started pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Container started 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Readiness probe error: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Readiness probe error: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Readiness probe error: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Readiness probe failed: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Readiness probe error: Get "http://10.132.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Readiness probe failed: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Readiness probe error: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Readiness probe failed: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Readiness probe error: Get "http://10.133.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Readiness probe failed: Get "http://10.132.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Readiness probe failed: Get "http://10.134.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Readiness probe failed: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Readiness probe error: Get "http://10.134.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Readiness probe failed: Get "http://10.133.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Readiness probe error: Get "http://10.134.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning ProbeError pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Readiness probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Readiness probe failed: Get "http://10.134.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 44m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Readiness probe error: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Readiness probe failed: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-b61ed672 Scaled up replica set payload-pre-processing-e2e-ait-b61ed672-65dd489d55 from 0 to 1 44m Normal Created pod/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-tqh66 Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-tqh66 Add eth0 [10.132.0.141/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-tqh66 Container started 44m Normal AddedInterface pod/payload-processing-e2e-derive-8b58192f-f48df6874-wbtnf Add eth0 [10.132.0.142/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-tqh66 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-b61ed672 Scaled up replica set payload-processing-e2e-ait-b61ed672-65b7c7796d from 0 to 1 44m Normal Started pod/payload-processing-e2e-derive-8b58192f-f48df6874-wbtnf Container started 44m Normal Created pod/payload-processing-e2e-derive-8b58192f-f48df6874-wbtnf Container created 44m Normal Pulled pod/payload-processing-e2e-derive-8b58192f-f48df6874-wbtnf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-b61ed672-65b7c7796d Created pod: payload-processing-e2e-ait-b61ed672-65b7c7796d-5g7zk 44m Normal Created pod/payload-pre-processing-e2e-ait-b61ed672-65dd489d55-tfgts Container created 44m Normal Pulled pod/payload-pre-processing-e2e-ait-b61ed672-65dd489d55-tfgts Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-b61ed672-65dd489d55 Error creating: pods "payload-pre-processing-e2e-ait-b61ed672-65dd489d55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b61ed672: serviceaccount "payload-processing-e2e-ait-b61ed672" not found 44m Warning FailedCreate replicaset/payload-processing-e2e-ait-b61ed672-65b7c7796d Error creating: pods "payload-processing-e2e-ait-b61ed672-65b7c7796d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b61ed672: serviceaccount "payload-processing-e2e-ait-b61ed672" not found 44m Normal AddedInterface pod/payload-pre-processing-e2e-ait-b61ed672-65dd489d55-tfgts Add eth0 [10.133.0.78/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-ait-b61ed672-65dd489d55-tfgts Container started 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-b61ed672-65dd489d55 Created pod: payload-pre-processing-e2e-ait-b61ed672-65dd489d55-tfgts 44m Normal Started pod/payload-processing-e2e-ait-b61ed672-65b7c7796d-5g7zk Container started 44m Normal Created pod/payload-processing-e2e-ait-b61ed672-65b7c7796d-5g7zk Container created 44m Normal AddedInterface pod/payload-processing-e2e-ait-b61ed672-65b7c7796d-5g7zk Add eth0 [10.132.0.144/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-ait-b61ed672-65b7c7796d-5g7zk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-8b58192f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-8b58192f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-b61ed672-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-b61ed672-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled up replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 4 to 5 44m Normal SuccessfulCreate replicaset/e2e-worker-w3-82b647-openshift-default-6747d4d57c Created pod: e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 44m Normal ScalingReplicaSet deployment/e2e-worker-w3-82b647-openshift-default Scaled up replica set e2e-worker-w3-82b647-openshift-default-6747d4d57c from 2 to 3 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-82b647-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 44m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg Add eth0 [10.132.0.145/23] from ovn-kubernetes 44m Normal Started pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 Container started 44m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg Container created 44m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-8b58192f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-8b58192f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedInterface pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 Add eth0 [10.132.0.146/23] from ovn-kubernetes 44m Normal Pulled pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg Container started 44m Normal Created pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 Container created 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-b61ed672-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-b61ed672-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Readiness probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Readiness probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg Startup probe error: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg Startup probe failed: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused 44m Warning ProbeError pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Readiness probe failed: Get "http://10.132.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Readiness probe error: Get "http://10.132.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Started pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Container started 43m Normal Created pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Container created 43m Normal Pulled pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Killing pod/payload-processing-e2e-ait-0660f91f-c4956954b-s2lkw Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-ait-0660f91f-6f55c5558c-gbhqt Stopping container payload-pre-processing 43m Warning ProbeError pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Startup probe error: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Startup probe failed: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-0660f91f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-0660f91f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedToRetrieveImagePullSecret pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Unable to retrieve some image pull secrets (e2e-ait-0660f91f-openshift-default-dockercfg-wl72z); attempting to pull the image may not succeed. 43m Normal ScalingReplicaSet deployment/e2e-migrate-837d7038-gw-openshift-default Scaled up replica set e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5 from 0 to 1 43m Normal AddedLabel gateway/e2e-migrate-837d7038-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-837d7038-gw 43m Normal SuccessfulCreate replicaset/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5 Created pod: e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 43m Normal AddedInterface pod/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 Add eth0 [10.134.0.78/23] from ovn-kubernetes 43m Normal Killing pod/e2e-ait-0660f91f-openshift-default-5b87f4869c-56djz Stopping container istio-proxy 43m Normal Killing pod/payload-pre-processing-e2e-derive-8b58192f-65f9c8cdf9-tqh66 Stopping container payload-pre-processing 43m Normal Killing pod/payload-processing-e2e-derive-8b58192f-f48df6874-wbtnf Stopping container payload-processing 43m Normal Killing pod/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 Stopping container istio-proxy 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c Error creating: pods "payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-837d7038: serviceaccount "payload-processing-e2e-migrate-837d7038" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-migrate-837d7038-569cf58996 Error creating: pods "payload-processing-e2e-migrate-837d7038-569cf58996-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-837d7038: serviceaccount "payload-processing-e2e-migrate-837d7038" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-837d7038 Scaled up replica set payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-837d7038 Scaled up replica set payload-processing-e2e-migrate-837d7038-569cf58996 from 0 to 1 43m Normal Pulled pod/payload-processing-e2e-migrate-837d7038-569cf58996-hz8jm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c Created pod: payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-sql7t 43m Normal Killing pod/payload-processing-e2e-ait-b61ed672-65b7c7796d-5g7zk Stopping container payload-processing 43m Normal AddedInterface pod/payload-processing-e2e-migrate-837d7038-569cf58996-hz8jm Add eth0 [10.133.0.80/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-sql7t Container started 43m Warning Unhealthy pod/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-837d7038-569cf58996 Created pod: payload-processing-e2e-migrate-837d7038-569cf58996-hz8jm 43m Normal Created pod/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-sql7t Container created 43m Normal Pulled pod/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-sql7t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning ProbeError pod/e2e-derive-8b58192f-openshift-default-754c75cb97-qdm78 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-sql7t Add eth0 [10.133.0.79/23] from ovn-kubernetes 43m Normal Killing pod/payload-pre-processing-e2e-ait-b61ed672-65dd489d55-tfgts Stopping container payload-pre-processing 43m Normal Created pod/payload-processing-e2e-migrate-837d7038-569cf58996-hz8jm Container created 43m Normal Started pod/payload-processing-e2e-migrate-837d7038-569cf58996-hz8jm Container started 43m Normal Killing pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Stopping container istio-proxy 43m Warning ProbeError pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-ait-b61ed672-openshift-default-6cb675d767-jfnwl Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-xq4mm 43m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-xq4mm Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 43m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-xq4mm Add eth0 [10.134.0.80/23] from ovn-kubernetes 43m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-xq4mm Container created 43m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-xq4mm Container started 43m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-xq4mm Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.579s (2.579s including waiting). Image size: 185476477 bytes. 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-837d7038-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-837d7038-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Created pod/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 Container created 43m Normal Started pod/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 Container started 43m Normal Pulled pod/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning Unhealthy pod/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 43m Warning ProbeError pod/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 42m Normal Killing pod/payload-pre-processing-e2e-migrate-837d7038-6c886b9b8c-sql7t Stopping container payload-pre-processing 42m Normal Killing pod/payload-processing-e2e-migrate-837d7038-569cf58996-hz8jm Stopping container payload-processing 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-837d7038-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-18a189-openshift-default Scaled up replica set e2e-shared-a-w2-18a189-openshift-default-76698f9dc9 from 0 to 1 42m Normal AddedLabel gateway/e2e-shared-a-w2-18a189 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-18a189 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9 Created pod: e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-837d7038-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal AddedInterface pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s Add eth0 [10.132.0.155/23] from ovn-kubernetes 42m Normal Killing pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Deleted pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-w5z5l 42m Normal Killing pod/e2e-migrate-837d7038-gw-openshift-default-75ffb8dbd5-jd7l4 Stopping container istio-proxy 42m Normal SuccessfulCreate replicaset/e2e-mt-08065c8e-openshift-default-6f958bfddf Created pod: e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 42m Normal AddedLabel gateway/e2e-mt-08065c8e Added label istio.io/rev=openshift-gateway to gateway e2e-mt-08065c8e 42m Normal ScalingReplicaSet deployment/e2e-mt-08065c8e-openshift-default Scaled up replica set e2e-mt-08065c8e-openshift-default-6f958bfddf from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979 Created pod: payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-18a189-d4495f868 Error creating: pods "payload-processing-e2e-shared-a-w2-18a189-d4495f868-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-18a189: serviceaccount "payload-processing-e2e-shared-a-w2-18a189" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-18a189: serviceaccount "payload-processing-e2e-shared-a-w2-18a189" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-18a189 Scaled up replica set payload-processing-e2e-shared-a-w2-18a189-d4495f868 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-18a189 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-18a189-57d89979 from 0 to 1 42m Normal AddedInterface pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Add eth0 [10.132.0.156/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk Add eth0 [10.133.0.81/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f Created pod: e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-eacea3-openshift-default Scaled up replica set e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f from 0 to 1 42m Normal AddedLabel gateway/e2e-shared-a-w0-eacea3 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-eacea3 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk Container created 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk Container started 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-18a189-d4495f868 Created pod: payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 42m Normal SuccessfulCreate replicaset/e2e-mt-fb3370cb-openshift-default-7488cd865 Created pod: e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 42m Normal AddedLabel gateway/e2e-mt-fb3370cb Added label istio.io/rev=openshift-gateway to gateway e2e-mt-fb3370cb 42m Normal ScalingReplicaSet deployment/e2e-mt-fb3370cb-openshift-default Scaled up replica set e2e-mt-fb3370cb-openshift-default-7488cd865 from 0 to 1 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 Add eth0 [10.132.0.157/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 Container started 42m Normal AddedInterface pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Add eth0 [10.132.0.158/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 Container created 42m Normal AddedInterface pod/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 Add eth0 [10.133.0.82/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-08065c8e Scaled up replica set payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7 from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-mt-08065c8e-68f98d88c4 Error creating: pods "payload-processing-e2e-mt-08065c8e-68f98d88c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-08065c8e: serviceaccount "payload-processing-e2e-mt-08065c8e" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7 Error creating: pods "payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-08065c8e: serviceaccount "payload-processing-e2e-mt-08065c8e" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-08065c8e Scaled up replica set payload-processing-e2e-mt-08065c8e-68f98d88c4 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7 Created pod: payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-m6bds 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-08065c8e-68f98d88c4 Created pod: payload-processing-e2e-mt-08065c8e-68f98d88c4-svnks 42m Normal Started pod/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-m6bds Container started 42m Normal Pulled pod/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-m6bds Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Pulled pod/payload-processing-e2e-mt-08065c8e-68f98d88c4-svnks Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-mt-08065c8e-68f98d88c4-svnks Add eth0 [10.132.0.160/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-mt-08065c8e-68f98d88c4-svnks Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-m6bds Add eth0 [10.132.0.159/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-m6bds Container created 42m Normal Created pod/payload-processing-e2e-mt-08065c8e-68f98d88c4-svnks Container created 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947 Created pod: payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-fb3370cb Scaled up replica set payload-processing-e2e-mt-fb3370cb-5cd6d77f9b from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-fb3370cb Scaled up replica set payload-pre-processing-e2e-mt-fb3370cb-5569c6b947 from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b Error creating: pods "payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-fb3370cb: serviceaccount "payload-processing-e2e-mt-fb3370cb" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b Created pod: payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947 Error creating: pods "payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-fb3370cb: serviceaccount "payload-processing-e2e-mt-fb3370cb" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7 Created pod: payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-eacea3 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-eacea3: serviceaccount "payload-processing-e2e-shared-a-w0-eacea3" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd Error creating: pods "payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-eacea3: serviceaccount "payload-processing-e2e-shared-a-w0-eacea3" not found 42m Normal AddedInterface pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 Add eth0 [10.133.0.83/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd Created pod: payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq 42m Normal Pulled pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 Container created 42m Normal Started pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 Container started 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-eacea3 Scaled up replica set payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd from 0 to 1 42m Normal AddedInterface pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g Add eth0 [10.132.0.161/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l Container created 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l Add eth0 [10.133.0.84/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq Container started 42m Normal Started pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g Container started 42m Normal Created pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g Container created 42m Normal Pulled pod/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq Container created 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq Add eth0 [10.133.0.85/23] from ovn-kubernetes 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-fb3370cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-fb3370cb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled up replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 5 to 10 42m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft 42m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk 42m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c (combined from similar events): Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp 42m Normal SuccessfulCreate replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Created pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm 42m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c Add eth0 [10.134.0.83/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm Container created 42m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm Container started 42m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Created pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww 42m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Created pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c 42m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp Add eth0 [10.132.0.163/23] from ovn-kubernetes 42m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk Container started 42m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm Add eth0 [10.133.0.88/23] from ovn-kubernetes 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 42m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz Add eth0 [10.132.0.164/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz Container created 42m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz Container started 42m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk Container created 42m Normal Pulled pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk Add eth0 [10.134.0.82/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba3d7b-openshift-default Scaled up replica set e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 from 3 to 5 42m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp Container started 42m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp Container created 42m Normal AddedInterface pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft Add eth0 [10.133.0.89/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft Container started 42m Normal Created pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft Container created 42m Normal Started pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww Container started 42m Normal Created pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww Container created 42m Normal Pulled pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c Container started 42m Normal Created pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c Container created 42m Normal AddedInterface pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww Add eth0 [10.132.0.165/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz Startup probe error: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz Startup probe failed: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww Startup probe failed: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww Startup probe error: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-2be695-openshift-default Scaled up replica set e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 from 0 to 1 42m Normal AddedLabel gateway/e2e-shared-b-w2-2be695 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-2be695 42m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Add eth0 [10.132.0.166/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-2be695 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7 Created pod: payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-2be695: serviceaccount "payload-processing-e2e-shared-b-w2-2be695" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68 Created pod: payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68 Error creating: pods "payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-2be695: serviceaccount "payload-processing-e2e-shared-b-w2-2be695" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-2be695 Scaled up replica set payload-processing-e2e-shared-b-w2-2be695-55dcd95b68 from 0 to 1 41m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Readiness probe error: Get "http://10.132.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Readiness probe failed: Get "http://10.132.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 Add eth0 [10.132.0.167/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 Container created 41m Normal Created pod/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 Container created 41m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 Container started 41m Normal Started pod/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 Container started 41m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 Add eth0 [10.132.0.168/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 Container started 41m Normal Pulled pod/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 Container created 41m Normal AddedLabel gateway/e2e-shared-b-w0-69dc90 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-69dc90 41m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-69dc90-openshift-default-df55b565f Created pod: e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj 41m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-69dc90-openshift-default Scaled up replica set e2e-shared-b-w0-69dc90-openshift-default-df55b565f from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-api-a-17d855-openshift-default-599df4fbdf Created pod: e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db 41m Normal AddedInterface pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Add eth0 [10.134.0.85/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/e2e-api-a-17d855-openshift-default Scaled up replica set e2e-api-a-17d855-openshift-default-599df4fbdf from 0 to 1 41m Normal AddedLabel gateway/e2e-api-a-17d855 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-17d855 41m Normal AddedInterface pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Add eth0 [10.132.0.169/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Readiness probe error: Get "http://10.132.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Readiness probe failed: Get "http://10.132.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-17d855 Scaled up replica set payload-pre-processing-e2e-api-a-17d855-7579cb887 from 0 to 1 41m Warning ProbeError pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Readiness probe error: Get "http://10.132.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Readiness probe failed: Get "http://10.132.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-17d855-7579cb887 Error creating: pods "payload-pre-processing-e2e-api-a-17d855-7579cb887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-17d855: serviceaccount "payload-processing-e2e-api-a-17d855" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-17d855 Scaled up replica set payload-processing-e2e-api-a-17d855-7cd78798fd from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-17d855-7579cb887 Created pod: payload-pre-processing-e2e-api-a-17d855-7579cb887-tl257 41m Warning FailedCreate replicaset/payload-processing-e2e-api-a-17d855-7cd78798fd Error creating: pods "payload-processing-e2e-api-a-17d855-7cd78798fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-17d855: serviceaccount "payload-processing-e2e-api-a-17d855" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-17d855-7cd78798fd Created pod: payload-processing-e2e-api-a-17d855-7cd78798fd-tdv5f 41m Normal AddedInterface pod/payload-processing-e2e-api-a-17d855-7cd78798fd-tdv5f Add eth0 [10.133.0.90/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-api-a-17d855-7cd78798fd-tdv5f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-api-a-17d855-7cd78798fd-tdv5f Container started 41m Normal Created pod/payload-processing-e2e-api-a-17d855-7cd78798fd-tdv5f Container created 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-17d855-7579cb887-tl257 Add eth0 [10.133.0.91/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-api-a-17d855-7579cb887-tl257 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-api-a-17d855-7579cb887-tl257 Container created 41m Normal Started pod/payload-pre-processing-e2e-api-a-17d855-7579cb887-tl257 Container started 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8 Created pod: payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db 41m Normal Started pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Container started 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-18a189-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-18a189-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Pulled pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Container created 41m Normal Pulled pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Started pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s Container started 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-69dc90 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5 from 0 to 1 41m Normal Pulled pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Container created 41m Normal Started pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Container started 41m Normal Created pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s Container created 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5 Created pod: payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8 Error creating: pods "payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-69dc90: serviceaccount "payload-processing-e2e-shared-b-w0-69dc90" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-69dc90 Scaled up replica set payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-69dc90: serviceaccount "payload-processing-e2e-shared-b-w0-69dc90" not found 41m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db Add eth0 [10.133.0.93/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db Container started 41m Normal Created pod/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db Container created 41m Normal Pulled pod/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h Add eth0 [10.132.0.171/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h Container created 41m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h Container started 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-08065c8e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-08065c8e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-eacea3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-eacea3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning Unhealthy pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Startup probe failed: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Startup probe error: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Startup probe failed: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Startup probe error: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused... 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-fb3370cb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-fb3370cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning ProbeError pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s Startup probe error: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s Startup probe failed: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-18a189-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-18a189-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulCreate replicaset/e2e-api-a-260268-openshift-default-d64f98dbf Created pod: e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 41m Normal AddedLabel gateway/e2e-api-a-260268 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-260268 41m Normal ScalingReplicaSet deployment/e2e-api-a-260268-openshift-default Scaled up replica set e2e-api-a-260268-openshift-default-d64f98dbf from 0 to 1 40m Normal AddedInterface pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Add eth0 [10.132.0.172/23] from ovn-kubernetes 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-08065c8e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-08065c8e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-eacea3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-eacea3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-260268 Scaled up replica set payload-processing-e2e-api-a-260268-5558bb6674 from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-api-a-260268-5558bb6674 Error creating: pods "payload-processing-e2e-api-a-260268-5558bb6674-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-260268: serviceaccount "payload-processing-e2e-api-a-260268" not found 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-260268-89664cb59 Error creating: pods "payload-pre-processing-e2e-api-a-260268-89664cb59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-260268: serviceaccount "payload-processing-e2e-api-a-260268" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-260268 Scaled up replica set payload-pre-processing-e2e-api-a-260268-89664cb59 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-260268-5558bb6674 Created pod: payload-processing-e2e-api-a-260268-5558bb6674-bwd6f 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-260268-89664cb59 Created pod: payload-pre-processing-e2e-api-a-260268-89664cb59-4p45z 40m Normal Pulled pod/payload-pre-processing-e2e-api-a-260268-89664cb59-4p45z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-260268-89664cb59-4p45z Add eth0 [10.132.0.174/23] from ovn-kubernetes 40m Normal AddedInterface pod/payload-processing-e2e-api-a-260268-5558bb6674-bwd6f Add eth0 [10.132.0.173/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-api-a-260268-5558bb6674-bwd6f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-processing-e2e-api-a-260268-5558bb6674-bwd6f Container created 40m Normal Started pod/payload-processing-e2e-api-a-260268-5558bb6674-bwd6f Container started 40m Normal Started pod/payload-pre-processing-e2e-api-a-260268-89664cb59-4p45z Container started 40m Normal Created pod/payload-pre-processing-e2e-api-a-260268-89664cb59-4p45z Container created 40m Normal Pulled pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Container started 40m Normal Created pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Container created 40m Normal AddedLabel gateway/e2e-api-b-288da6 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-288da6 40m Normal AddedInterface pod/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj Add eth0 [10.134.0.88/23] from ovn-kubernetes 40m Normal Created pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Container created 40m Normal Pulled pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-api-b-288da6-openshift-default-68599494b9 Created pod: e2e-api-b-288da6-openshift-default-68599494b9-5kwfj 40m Normal Started pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Container started 40m Normal ScalingReplicaSet deployment/e2e-api-b-288da6-openshift-default Scaled up replica set e2e-api-b-288da6-openshift-default-68599494b9 from 0 to 1 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-69dc90-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-69dc90-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning Unhealthy pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-288da6 Scaled up replica set payload-pre-processing-e2e-api-b-288da6-7d5db4cc67 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67 Created pod: payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-9xs79 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67 Error creating: pods "payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-288da6: serviceaccount "payload-processing-e2e-api-b-288da6" not found 40m Warning FailedCreate replicaset/payload-processing-e2e-api-b-288da6-b54cbc654 Error creating: pods "payload-processing-e2e-api-b-288da6-b54cbc654-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-288da6: serviceaccount "payload-processing-e2e-api-b-288da6" not found 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-288da6 Scaled up replica set payload-processing-e2e-api-b-288da6-b54cbc654 from 0 to 1 40m Normal AddedInterface pod/payload-processing-e2e-api-b-288da6-b54cbc654-wx2zn Add eth0 [10.133.0.95/23] from ovn-kubernetes 40m Normal Started pod/payload-processing-e2e-api-b-288da6-b54cbc654-wx2zn Container started 40m Normal Pulled pod/payload-processing-e2e-api-b-288da6-b54cbc654-wx2zn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-288da6-b54cbc654 Created pod: payload-processing-e2e-api-b-288da6-b54cbc654-wx2zn 40m Normal Created pod/payload-processing-e2e-api-b-288da6-b54cbc654-wx2zn Container created 40m Warning Unhealthy pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Startup probe failed: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused 40m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-9xs79 Add eth0 [10.132.0.177/23] from ovn-kubernetes 40m Normal Started pod/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-9xs79 Container started 40m Normal Pulled pod/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-9xs79 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-9xs79 Container created 40m Warning ProbeError pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Startup probe error: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused... 40m Normal Killing pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 Stopping container payload-processing 40m Normal AddedLabel gateway/e2e-inf-a-a3204e Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-a3204e 40m Normal SuccessfulCreate replicaset/e2e-inf-a-a3204e-openshift-default-57fcb546f7 Created pod: e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 40m Normal ScalingReplicaSet deployment/e2e-inf-a-a3204e-openshift-default Scaled up replica set e2e-inf-a-a3204e-openshift-default-57fcb546f7 from 0 to 1 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-17d855-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-17d855-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal AddedInterface pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Add eth0 [10.132.0.178/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-a3204e-756d475586 Created pod: payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-a3204e Scaled up replica set payload-processing-e2e-inf-a-a3204e-756d475586 from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-a3204e-756d475586 Error creating: pods "payload-processing-e2e-inf-a-a3204e-756d475586-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a3204e: serviceaccount "payload-processing-e2e-inf-a-a3204e" not found 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5 Error creating: pods "payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a3204e: serviceaccount "payload-processing-e2e-inf-a-a3204e" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-a3204e Scaled up replica set payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5 Created pod: payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 40m Normal Pulled pod/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z Add eth0 [10.132.0.180/23] from ovn-kubernetes 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 Add eth0 [10.132.0.179/23] from ovn-kubernetes 40m Normal Created pod/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 Container created 40m Normal Started pod/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 Container started 40m Normal Pulled pod/payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z Container created 40m Normal Started pod/payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z Container started 40m Warning Unhealthy pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-69dc90-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-69dc90-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-17d855-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-17d855-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning Unhealthy pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-m77c7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal Killing pod/payload-pre-processing-e2e-mt-08065c8e-67fbc9b5b7-m6bds Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-mt-08065c8e-68f98d88c4-svnks Stopping container payload-processing 40m Warning Unhealthy pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-cvh5g Readiness probe failed: timeout: failed to connect service "10.132.0.161:50052" within 1s: context deadline exceeded 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Readiness probe failed: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Readiness probe error: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Readiness probe failed: Get "http://10.132.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Readiness probe error: Get "http://10.132.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Normal Started pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Container started 39m Normal Created pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Container created 39m Normal Pulled pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedLabel gateway/e2e-api-b-07410c Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-07410c 39m Normal ScalingReplicaSet deployment/e2e-api-b-07410c-openshift-default Scaled up replica set e2e-api-b-07410c-openshift-default-b6cc5f76f from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-api-b-07410c-openshift-default-b6cc5f76f Created pod: e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp 39m Normal AddedInterface pod/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp Add eth0 [10.132.0.183/23] from ovn-kubernetes 39m Normal AddedLabel gateway/e2e-api-a-13a3e2 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-13a3e2 39m Normal ScalingReplicaSet deployment/e2e-api-a-13a3e2-openshift-default Scaled up replica set e2e-api-a-13a3e2-openshift-default-5868ff964c from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-api-a-13a3e2-openshift-default-5868ff964c Created pod: e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw 39m Normal AddedInterface pod/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw Add eth0 [10.133.0.97/23] from ovn-kubernetes 39m Warning Unhealthy pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Startup probe failed: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused 39m Warning ProbeError pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Startup probe error: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused... 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-07410c Scaled up replica set payload-pre-processing-e2e-api-b-07410c-67d6df4667 from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-07410c-67d6df4667 Error creating: pods "payload-pre-processing-e2e-api-b-07410c-67d6df4667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-07410c: serviceaccount "payload-processing-e2e-api-b-07410c" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-07410c Scaled up replica set payload-processing-e2e-api-b-07410c-7598fb6b98 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-07410c-7598fb6b98 Created pod: payload-processing-e2e-api-b-07410c-7598fb6b98-8r4jm 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-260268-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-07410c-67d6df4667 Created pod: payload-pre-processing-e2e-api-b-07410c-67d6df4667-nwwnp 39m Warning FailedCreate replicaset/payload-processing-e2e-api-b-07410c-7598fb6b98 Error creating: pods "payload-processing-e2e-api-b-07410c-7598fb6b98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-07410c: serviceaccount "payload-processing-e2e-api-b-07410c" not found 39m Normal Started pod/payload-processing-e2e-api-b-07410c-7598fb6b98-8r4jm Container started 39m Normal Started pod/payload-pre-processing-e2e-api-b-07410c-67d6df4667-nwwnp Container started 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-260268-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-07410c-67d6df4667-nwwnp Add eth0 [10.132.0.184/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-api-b-07410c-67d6df4667-nwwnp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-processing-e2e-api-b-07410c-7598fb6b98-8r4jm Container created 39m Normal Pulled pod/payload-processing-e2e-api-b-07410c-7598fb6b98-8r4jm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-api-b-07410c-7598fb6b98-8r4jm Add eth0 [10.132.0.185/23] from ovn-kubernetes 39m Normal Created pod/payload-pre-processing-e2e-api-b-07410c-67d6df4667-nwwnp Container created 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c Error creating: pods "payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-13a3e2: serviceaccount "payload-processing-e2e-api-a-13a3e2" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-13a3e2 Scaled up replica set payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c Created pod: payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-6bhcg 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-6bhcg Add eth0 [10.133.0.98/23] from ovn-kubernetes 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-13a3e2-d4978667b Error creating: pods "payload-processing-e2e-api-a-13a3e2-d4978667b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-13a3e2: serviceaccount "payload-processing-e2e-api-a-13a3e2" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-13a3e2-d4978667b Created pod: payload-processing-e2e-api-a-13a3e2-d4978667b-txkrw 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-13a3e2 Scaled up replica set payload-processing-e2e-api-a-13a3e2-d4978667b from 0 to 1 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-6bhcg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-api-a-13a3e2-d4978667b-txkrw Container started 39m Normal Pulled pod/payload-processing-e2e-api-a-13a3e2-d4978667b-txkrw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-api-a-13a3e2-d4978667b-txkrw Add eth0 [10.133.0.99/23] from ovn-kubernetes 39m Normal Created pod/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-6bhcg Container created 39m Normal Created pod/payload-processing-e2e-api-a-13a3e2-d4978667b-txkrw Container created 39m Normal Started pod/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-6bhcg Container started 39m Normal Created pod/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj Container created 39m Normal Started pod/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj Container started 39m Normal Pulled pod/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 7; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Deleted pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p 39m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-w6w4p Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled down replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 10 to 7 39m Normal SuccessfulDelete replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Deleted pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl 39m Normal SuccessfulDelete replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Deleted pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 39m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-q9dm5 Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-j7lcl Stopping container istio-proxy 39m Normal Killing pod/e2e-mt-08065c8e-openshift-default-6f958bfddf-jqkv9 Stopping container istio-proxy 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-288da6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-288da6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-260268-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-260268-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal AddedLabel gateway/e2e-shared-a-w1-262230 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-262230 39m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-262230-openshift-default Scaled up replica set e2e-shared-a-w1-262230-openshift-default-9d45d8bbc from 0 to 1 39m Normal AddedInterface pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Add eth0 [10.133.0.100/23] from ovn-kubernetes 39m Warning Unhealthy pod/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 39m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc Created pod: e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp 39m Warning ProbeError pod/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-262230 Scaled up replica set payload-processing-e2e-shared-a-w1-262230-9f68f6684 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-262230 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-262230-9f68f6684 Created pod: payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-262230-9f68f6684 Error creating: pods "payload-processing-e2e-shared-a-w1-262230-9f68f6684-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-262230: serviceaccount "payload-processing-e2e-shared-a-w1-262230" not found 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-262230: serviceaccount "payload-processing-e2e-shared-a-w1-262230" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7 Created pod: payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm 39m Normal Killing pod/payload-pre-processing-e2e-api-a-17d855-7579cb887-tl257 Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-a-17d855-7cd78798fd-tdv5f Stopping container payload-processing 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm Add eth0 [10.132.0.188/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Pulled pod/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf Add eth0 [10.132.0.187/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm Container started 39m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm Container created 39m Normal Started pod/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf Container started 39m Normal Created pod/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf Container created 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b Created pod: payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-2686h 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-fb3370cb Scaled up replica set payload-processing-e2e-mt-fb3370cb-5cd6d77f9b from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947 Error creating: pods "payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-fb3370cb: serviceaccount "payload-processing-e2e-mt-fb3370cb" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-fb3370cb Scaled up replica set payload-pre-processing-e2e-mt-fb3370cb-5569c6b947 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947 Created pod: payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-j9fwr 39m Warning FailedCreate replicaset/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b Error creating: pods "payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-fb3370cb: serviceaccount "payload-processing-e2e-mt-fb3370cb" not found 39m Normal Pulled pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-j9fwr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-j9fwr Container created 39m Normal Started pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-j9fwr Container started 39m Normal Started pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-2686h Container started 39m Normal Created pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-2686h Container created 39m Normal Pulled pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-2686h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-j9fwr Add eth0 [10.132.0.189/23] from ovn-kubernetes 39m Normal AddedInterface pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-2686h Add eth0 [10.132.0.190/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-inf-b-7e233f-openshift-default-74c6dbbb65 Created pod: e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr 39m Normal AddedLabel gateway/e2e-inf-b-7e233f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-7e233f 39m Normal ScalingReplicaSet deployment/e2e-inf-b-7e233f-openshift-default Scaled up replica set e2e-inf-b-7e233f-openshift-default-74c6dbbb65 from 0 to 1 39m Normal AddedInterface pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr Add eth0 [10.132.0.191/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-7e233f-b5cfd5c8 Created pod: payload-processing-e2e-inf-b-7e233f-b5cfd5c8-8nwls 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c Created pod: payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-7e233f Scaled up replica set payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c Error creating: pods "payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-7e233f: serviceaccount "payload-processing-e2e-inf-b-7e233f" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-7e233f Scaled up replica set payload-processing-e2e-inf-b-7e233f-b5cfd5c8 from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-7e233f-b5cfd5c8 Error creating: pods "payload-processing-e2e-inf-b-7e233f-b5cfd5c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-7e233f: serviceaccount "payload-processing-e2e-inf-b-7e233f" not found 39m Normal Created pod/payload-processing-e2e-inf-b-7e233f-b5cfd5c8-8nwls Container created 39m Normal Created pod/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr Container created 39m Normal Pulled pod/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr Add eth0 [10.132.0.192/23] from ovn-kubernetes 39m Normal AddedInterface pod/payload-processing-e2e-inf-b-7e233f-b5cfd5c8-8nwls Add eth0 [10.132.0.193/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr Container started 39m Normal Started pod/payload-processing-e2e-inf-b-7e233f-b5cfd5c8-8nwls Container started 39m Normal Pulled pod/payload-processing-e2e-inf-b-7e233f-b5cfd5c8-8nwls Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 7; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-82b647-openshift-default New size: 2; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-82b647-openshift-default Scaled down replica set e2e-worker-w3-82b647-openshift-default-6747d4d57c from 3 to 2 39m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Deleted pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-9qftd 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Deleted pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-xjdpr 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Deleted pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-bqjqb 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-82b647-openshift-default-6747d4d57c Deleted pod: e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled down replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 10 to 7 39m Normal Killing pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-wlrz5 Stopping container istio-proxy 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-288da6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-288da6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 38m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-n6c2h 38m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-n6c2h Add eth0 [10.132.0.195/23] from ovn-kubernetes 38m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-n6c2h Container started 38m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-n6c2h Container created 38m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-n6c2h Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default New size: 7; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled down replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 10 to 7 38m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w6-c4847c-openshift-default Scaled down replica set e2e-worker-w6-c4847c-openshift-default-65bf677b7f from 10 to 7 38m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 7; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Deleted pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-d52rg 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Deleted pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-phpkv 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Deleted pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-cr9pd 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Deleted pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-x64vc 38m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Deleted pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-2vdm8 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Deleted pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-flxfn 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-262230-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-262230-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal SuccessfulCreate replicaset/e2e-api-b-da60de-openshift-default-64ff6c4bc Created pod: e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn 38m Normal Pulled pod/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw Container started 38m Normal Created pod/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw Container created 38m Normal ScalingReplicaSet deployment/e2e-api-b-da60de-openshift-default Scaled up replica set e2e-api-b-da60de-openshift-default-64ff6c4bc from 0 to 1 38m Normal AddedLabel gateway/e2e-api-b-da60de Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-da60de 38m Normal AddedInterface pod/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn Add eth0 [10.134.0.98/23] from ovn-kubernetes 38m Normal Created pod/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp Container created 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-da60de-dd5598cb7 Created pod: payload-pre-processing-e2e-api-b-da60de-dd5598cb7-fxnsc 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-da60de Scaled up replica set payload-processing-e2e-api-b-da60de-877f4976 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-da60de-877f4976 Created pod: payload-processing-e2e-api-b-da60de-877f4976-zv2gh 38m Normal Started pod/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp Container started 38m Warning FailedCreate replicaset/payload-processing-e2e-api-b-da60de-877f4976 Error creating: pods "payload-processing-e2e-api-b-da60de-877f4976-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-da60de: serviceaccount "payload-processing-e2e-api-b-da60de" not found 38m Warning ProbeError pod/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-da60de-dd5598cb7 Error creating: pods "payload-pre-processing-e2e-api-b-da60de-dd5598cb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-da60de: serviceaccount "payload-processing-e2e-api-b-da60de" not found 38m Warning Unhealthy pod/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 38m Normal Pulled pod/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-da60de Scaled up replica set payload-pre-processing-e2e-api-b-da60de-dd5598cb7 from 0 to 1 38m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-da60de-dd5598cb7-fxnsc Add eth0 [10.133.0.103/23] from ovn-kubernetes 38m Normal Started pod/payload-pre-processing-e2e-api-b-da60de-dd5598cb7-fxnsc Container started 38m Normal Pulled pod/payload-pre-processing-e2e-api-b-da60de-dd5598cb7-fxnsc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-api-b-da60de-dd5598cb7-fxnsc Container created 38m Normal Killing pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Stopping container istio-proxy 38m Normal Created pod/payload-processing-e2e-api-b-da60de-877f4976-zv2gh Container created 38m Normal Started pod/payload-processing-e2e-api-b-da60de-877f4976-zv2gh Container started 38m Normal AddedInterface pod/payload-processing-e2e-api-b-da60de-877f4976-zv2gh Add eth0 [10.132.0.197/23] from ovn-kubernetes 38m Normal Pulled pod/payload-processing-e2e-api-b-da60de-877f4976-zv2gh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Warning Unhealthy pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-api-a-17d855-openshift-default-599df4fbdf-rk5db Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Killing pod/payload-processing-e2e-api-a-260268-5558bb6674-bwd6f Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-api-a-260268-89664cb59-4p45z Stopping container payload-pre-processing 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-13a3e2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-13a3e2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning Unhealthy pod/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp Startup probe failed: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp Startup probe error: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused... 38m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Readiness probe error: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Readiness probe failed: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 Readiness probe failed: timeout: health rpc did not complete within 1s 38m Normal SuccessfulDelete replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Deleted pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 38m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled down replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 7 to 6 38m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-28fc5 Stopping container istio-proxy 38m Normal Killing pod/payload-pre-processing-e2e-api-b-288da6-7d5db4cc67-9xs79 Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-api-b-288da6-b54cbc654-wx2zn Stopping container payload-processing 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 6; reason: All metrics below target 38m Normal Started pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Container started 38m Normal Created pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Container created 38m Normal Pulled pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-07410c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-07410c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal AddedInterface pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Add eth0 [10.134.0.100/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-add1bb-openshift-default Scaled up replica set e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 Created pod: e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c 38m Normal AddedLabel gateway/e2e-shared-b-w1-add1bb Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-add1bb 38m Warning Unhealthy pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-13a3e2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-13a3e2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-da60de-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-da60de-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668 Error creating: pods "payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-add1bb: serviceaccount "payload-processing-e2e-shared-b-w1-add1bb" not found 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-add1bb Scaled up replica set payload-processing-e2e-shared-b-w1-add1bb-f9bccd668 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-add1bb Scaled up replica set payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-add1bb: serviceaccount "payload-processing-e2e-shared-b-w1-add1bb" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5 Created pod: payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw 38m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw Container created 38m Normal Pulled pod/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw Add eth0 [10.133.0.104/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 Container started 38m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw Container started 38m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 Add eth0 [10.133.0.105/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668 Created pod: payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 38m Normal Created pod/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 Container created 37m Normal Killing pod/payload-processing-e2e-mt-fb3370cb-5cd6d77f9b-2686h Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-mt-fb3370cb-5569c6b947-j9fwr Stopping container payload-pre-processing 37m Normal Created pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr Container created 37m Normal Pulled pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr Container started 37m Warning Unhealthy pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr Startup probe failed: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused 37m Warning ProbeError pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr Startup probe error: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused... 37m Normal Killing pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Stopping container istio-proxy 37m Warning ProbeError pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-api-a-260268-openshift-default-d64f98dbf-6rrh7 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-7e233f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-7e233f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-262230-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-262230-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal Pulled pod/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Readiness probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused 37m Normal Killing pod/payload-processing-e2e-api-b-07410c-7598fb6b98-8r4jm Stopping container payload-processing 37m Normal Pulled pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Killing pod/payload-pre-processing-e2e-api-b-07410c-67d6df4667-nwwnp Stopping container payload-pre-processing 37m Normal Started pod/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn Container started 37m Normal Started pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Container started 37m Warning ProbeError pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Readiness probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused... 37m Normal Created pod/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn Container created 37m Normal Created pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Container created 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-fb3370cb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal ScalingReplicaSet deployment/e2e-mt-fb3370cb-openshift-default Scaled up replica set e2e-mt-fb3370cb-openshift-default-7488cd865 from 1 to 2 37m Normal SuccessfulCreate replicaset/e2e-mt-fb3370cb-openshift-default-7488cd865 Created pod: e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm 37m Normal Pulled pod/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm Add eth0 [10.132.0.201/23] from ovn-kubernetes 37m Normal Created pod/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm Container created 37m Normal Started pod/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm Container started 37m Warning ProbeError pod/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused 37m Warning ProbeError pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Startup probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Startup probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Deleted pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 37m Warning ProbeError pod/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm Startup probe error: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused... 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 5; reason: All metrics below target 37m Warning Unhealthy pod/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm Startup probe failed: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused 37m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-n25b9 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled down replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 6 to 5 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-7e233f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-7e233f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a3204e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a3204e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal Killing pod/e2e-api-b-288da6-openshift-default-68599494b9-5kwfj Stopping container istio-proxy 37m Normal SuccessfulCreate replicaset/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f Created pod: e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w 37m Normal AddedLabel gateway/e2e-inf-a-52f1cb Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-52f1cb 37m Normal ScalingReplicaSet deployment/e2e-inf-a-52f1cb-openshift-default Scaled up replica set e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f from 0 to 1 37m Normal AddedInterface pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w Add eth0 [10.134.0.104/23] from ovn-kubernetes 37m Normal Killing pod/payload-pre-processing-e2e-api-a-13a3e2-5c5445bb8c-6bhcg Stopping container payload-pre-processing 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Deleted pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp 37m Normal Killing pod/payload-processing-e2e-api-a-13a3e2-d4978667b-txkrw Stopping container payload-processing 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Deleted pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz 37m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6wplp Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Deleted pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled down replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 10 to 7 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 7; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-qtsxg Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-ldmwz Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default New size: 4; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba3d7b-openshift-default Scaled down replica set e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 from 5 to 4 36m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Deleted pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-xkq4c 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a3204e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a3204e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f Created pod: payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f Error creating: pods "payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-52f1cb: serviceaccount "payload-processing-e2e-inf-a-52f1cb" not found 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-52f1cb-7f8d487558 Error creating: pods "payload-processing-e2e-inf-a-52f1cb-7f8d487558-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-52f1cb: serviceaccount "payload-processing-e2e-inf-a-52f1cb" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-52f1cb Scaled up replica set payload-pre-processing-e2e-inf-a-52f1cb-74c68749f from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-52f1cb Scaled up replica set payload-processing-e2e-inf-a-52f1cb-7f8d487558 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-52f1cb-7f8d487558 Created pod: payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr 36m Normal Pulled pod/payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps Container created 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps Add eth0 [10.132.0.204/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr Add eth0 [10.132.0.205/23] from ovn-kubernetes 36m Normal Started pod/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps Container started 36m Normal Started pod/payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr Container started 36m Normal Created pod/payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr Container created 36m Normal Killing pod/e2e-mt-fb3370cb-openshift-default-7488cd865-zrth6 Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-fb3370cb-openshift-default-7488cd865-pzhwm Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-inf-a-e92dab-openshift-default Scaled up replica set e2e-inf-a-e92dab-openshift-default-55f9bdc949 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-inf-a-e92dab-openshift-default-55f9bdc949 Created pod: e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 36m Normal AddedLabel gateway/e2e-inf-a-e92dab Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-e92dab 36m Normal AddedInterface pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Add eth0 [10.134.0.105/23] from ovn-kubernetes 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-da60de-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-da60de-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5 Error creating: pods "payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e92dab: serviceaccount "payload-processing-e2e-inf-a-e92dab" not found 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-e92dab-76bcb94f8b Error creating: pods "payload-processing-e2e-inf-a-e92dab-76bcb94f8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e92dab: serviceaccount "payload-processing-e2e-inf-a-e92dab" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-e92dab-76bcb94f8b Created pod: payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-e92dab Scaled up replica set payload-processing-e2e-inf-a-e92dab-76bcb94f8b from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-e92dab Scaled up replica set payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5 Created pod: payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc 36m Normal Created pod/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc Container created 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc Add eth0 [10.132.0.207/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 Add eth0 [10.132.0.208/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 Container started 36m Normal Pulled pod/payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 Container created 36m Normal Created pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Container created 36m Normal Started pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Container started 36m Normal Pulled pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedLabel gateway/e2e-shared-a-w5-64f472 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-64f472 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-64f472-openshift-default Scaled up replica set e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c Created pod: e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj 36m Normal AddedInterface pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Add eth0 [10.132.0.209/23] from ovn-kubernetes 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-add1bb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-add1bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning Unhealthy pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6 Error creating: pods "payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-64f472: serviceaccount "payload-processing-e2e-shared-a-w5-64f472" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-64f472 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf Created pod: payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf Error creating: pods "payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-64f472: serviceaccount "payload-processing-e2e-shared-a-w5-64f472" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-64f472 Scaled up replica set payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6 from 0 to 1 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn Add eth0 [10.132.0.210/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6 Created pod: payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq 36m Normal Started pod/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq Container started 36m Normal Pulled pod/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq Container created 36m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn Container created 36m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq Add eth0 [10.132.0.211/23] from ovn-kubernetes 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-52f1cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-52f1cb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn Container started 36m Normal SuccessfulCreate replicaset/e2e-inf-b-7e233f-openshift-default-74c6dbbb65 Created pod: e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l 36m Normal AddedInterface pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l Add eth0 [10.134.0.106/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/e2e-inf-b-7e233f-openshift-default Scaled up replica set e2e-inf-b-7e233f-openshift-default-74c6dbbb65 from 1 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-7e233f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal Started pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l Container started 36m Normal Pulled pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l Container created 36m Warning ProbeError pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-add1bb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-add1bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-e92dab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-e92dab-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal Killing pod/e2e-api-b-07410c-openshift-default-b6cc5f76f-fphzp Stopping container istio-proxy 36m Normal SuccessfulCreate replicaset/e2e-inf-a-4dde48-openshift-default-7599b7cfcc Created pod: e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr 36m Normal ScalingReplicaSet deployment/e2e-inf-a-4dde48-openshift-default Scaled up replica set e2e-inf-a-4dde48-openshift-default-7599b7cfcc from 0 to 1 36m Normal AddedLabel gateway/e2e-inf-a-4dde48 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-4dde48 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-64f472-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-64f472-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal Killing pod/e2e-api-a-13a3e2-openshift-default-5868ff964c-vbcvw Stopping container istio-proxy 36m Normal AddedInterface pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Add eth0 [10.132.0.212/23] from ovn-kubernetes 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7 Error creating: pods "payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4dde48: serviceaccount "payload-processing-e2e-inf-a-4dde48" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7 Created pod: payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-4dde48 Scaled up replica set payload-processing-e2e-inf-a-4dde48-8f44b4878 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-4dde48 Scaled up replica set payload-pre-processing-e2e-inf-a-4dde48-745c9456d7 from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-4dde48-8f44b4878 Error creating: pods "payload-processing-e2e-inf-a-4dde48-8f44b4878-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4dde48: serviceaccount "payload-processing-e2e-inf-a-4dde48" not found 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 Add eth0 [10.133.0.109/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 Container started 36m Normal Pulled pod/payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 Add eth0 [10.133.0.108/23] from ovn-kubernetes 36m Normal Created pod/payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 Container created 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-4dde48-8f44b4878 Created pod: payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 36m Normal Started pod/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 Container started 36m Normal Created pod/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 Container created 35m Normal Killing pod/payload-processing-e2e-api-b-da60de-877f4976-zv2gh Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-api-b-da60de-dd5598cb7-fxnsc Stopping container payload-pre-processing 35m Warning Unhealthy pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Readiness probe failed: Get "http://10.134.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning ProbeError pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Readiness probe error: Get "http://10.134.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal Started pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Container started 35m Normal Created pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Container created 35m Normal Pulled pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w Container created 35m Normal Started pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w Container started 35m Warning Unhealthy pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w Startup probe error: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w Startup probe failed: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused 35m Normal AddedLabel gateway/e2e-inf-b-687471 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-687471 35m Normal SuccessfulCreate replicaset/e2e-inf-b-687471-openshift-default-79cdb54bc5 Created pod: e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh 35m Normal ScalingReplicaSet deployment/e2e-inf-b-687471-openshift-default Scaled up replica set e2e-inf-b-687471-openshift-default-79cdb54bc5 from 0 to 1 35m Normal AddedLabel gateway/e2e-inf-a-82e9bf Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-82e9bf 35m Normal AddedInterface pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh Add eth0 [10.132.0.214/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr Add eth0 [10.133.0.111/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/e2e-inf-a-82e9bf-openshift-default Scaled up replica set e2e-inf-a-82e9bf-openshift-default-666946c759 from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-inf-a-82e9bf-openshift-default-666946c759 Created pod: e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr 35m Normal SuccessfulCreate replicaset/e2e-inf-b-56d1d9-openshift-default-5f88488bc9 Created pod: e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 35m Normal ScalingReplicaSet deployment/e2e-inf-b-56d1d9-openshift-default Scaled up replica set e2e-inf-b-56d1d9-openshift-default-5f88488bc9 from 0 to 1 35m Normal AddedLabel gateway/e2e-inf-b-56d1d9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-56d1d9 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-687471-57fb85bd67 Error creating: pods "payload-pre-processing-e2e-inf-b-687471-57fb85bd67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-687471: serviceaccount "payload-processing-e2e-inf-b-687471" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-687471-57fb85bd67 Created pod: payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-687471-74d85f5bdd Created pod: payload-processing-e2e-inf-b-687471-74d85f5bdd-qw568 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-687471 Scaled up replica set payload-pre-processing-e2e-inf-b-687471-57fb85bd67 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-687471 Scaled up replica set payload-processing-e2e-inf-b-687471-74d85f5bdd from 0 to 1 35m Normal AddedInterface pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Add eth0 [10.132.0.215/23] from ovn-kubernetes 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-687471-74d85f5bdd Error creating: pods "payload-processing-e2e-inf-b-687471-74d85f5bdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-687471: serviceaccount "payload-processing-e2e-inf-b-687471" not found 35m Normal Pulled pod/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w Container started 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w Add eth0 [10.133.0.113/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w Container created 35m Normal AddedInterface pod/payload-processing-e2e-inf-b-687471-74d85f5bdd-qw568 Add eth0 [10.132.0.217/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-inf-b-687471-74d85f5bdd-qw568 Container started 35m Normal Created pod/payload-processing-e2e-inf-b-687471-74d85f5bdd-qw568 Container created 35m Normal Pulled pod/payload-processing-e2e-inf-b-687471-74d85f5bdd-qw568 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd Created pod: payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-82e9bf Scaled up replica set payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf Created pod: payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf Error creating: pods "payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-82e9bf: serviceaccount "payload-processing-e2e-inf-a-82e9bf" not found 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd Error creating: pods "payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-82e9bf: serviceaccount "payload-processing-e2e-inf-a-82e9bf" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-82e9bf Scaled up replica set payload-processing-e2e-inf-a-82e9bf-7f866bf6dd from 0 to 1 35m Normal AddedInterface pod/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p Add eth0 [10.133.0.114/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p Container started 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht Add eth0 [10.132.0.218/23] from ovn-kubernetes 35m Normal Created pod/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p Container created 35m Normal Pulled pod/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht Container created 35m Normal Started pod/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht Container started 35m Normal Pulled pod/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-56d1d9 Scaled up replica set payload-processing-e2e-inf-b-56d1d9-59985c8c84 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-56d1d9-59985c8c84 Created pod: payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-56d1d9-59985c8c84 Error creating: pods "payload-processing-e2e-inf-b-56d1d9-59985c8c84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-56d1d9: serviceaccount "payload-processing-e2e-inf-b-56d1d9" not found 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc Error creating: pods "payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-56d1d9: serviceaccount "payload-processing-e2e-inf-b-56d1d9" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc Created pod: payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-56d1d9 Scaled up replica set payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc from 0 to 1 35m Normal AddedInterface pod/payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b Add eth0 [10.132.0.219/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 Add eth0 [10.133.0.115/23] from ovn-kubernetes 35m Normal Pulled pod/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 Container started 35m Normal Created pod/payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b Container created 35m Normal Started pod/payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b Container started 35m Normal AddedLabel gateway/e2e-shared-b-w5-29145c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-29145c 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f Created pod: e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-29145c-openshift-default Scaled up replica set e2e-shared-b-w5-29145c-openshift-default-56b4496f7f from 0 to 1 35m Normal AddedInterface pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Add eth0 [10.134.0.108/23] from ovn-kubernetes 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-52f1cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-52f1cb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7 Created pod: payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-29145c Scaled up replica set payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7 from 0 to 1 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-29145c: serviceaccount "payload-processing-e2e-shared-b-w5-29145c" not found 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-29145c-677554878d Error creating: pods "payload-processing-e2e-shared-b-w5-29145c-677554878d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-29145c: serviceaccount "payload-processing-e2e-shared-b-w5-29145c" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-29145c Scaled up replica set payload-processing-e2e-shared-b-w5-29145c-677554878d from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-29145c-677554878d Created pod: payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 35m Normal Created pod/payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 Container created 35m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 Container started 35m Normal Pulled pod/payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 Add eth0 [10.133.0.116/23] from ovn-kubernetes 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt Add eth0 [10.132.0.221/23] from ovn-kubernetes 35m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt Container started 35m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt Container created 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-e92dab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-e92dab-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning Unhealthy pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Readiness probe failed: Get "http://10.132.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning ProbeError pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Readiness probe error: Get "http://10.132.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal Pulled pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Container created 35m Normal Started pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Container started 34m Warning Unhealthy pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Startup probe failed: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Startup probe error: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused... 34m Normal Killing pod/payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 Stopping container payload-pre-processing 34m Normal Killing pod/e2e-api-b-da60de-openshift-default-64ff6c4bc-8ffkn Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-inf-a-50d078-openshift-default Scaled up replica set e2e-inf-a-50d078-openshift-default-c45b594b7 from 0 to 1 34m Normal AddedLabel gateway/e2e-inf-a-50d078 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-50d078 34m Normal SuccessfulCreate replicaset/e2e-inf-a-50d078-openshift-default-c45b594b7 Created pod: e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j 34m Normal AddedInterface pod/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j Add eth0 [10.133.0.117/23] from ovn-kubernetes 34m Normal AddedLabel gateway/e2e-inf-b-34e896 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-34e896 34m Normal ScalingReplicaSet deployment/e2e-inf-b-34e896-openshift-default Scaled up replica set e2e-inf-b-34e896-openshift-default-6c7475f655 from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-inf-b-34e896-openshift-default-6c7475f655 Created pod: e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 34m Normal AddedInterface pod/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 Add eth0 [10.132.0.223/23] from ovn-kubernetes 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-64f472-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-64f472-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-50d078 Scaled up replica set payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9 from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-50d078 Scaled up replica set payload-processing-e2e-inf-a-50d078-58ffdcc78d from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-50d078-58ffdcc78d Created pod: payload-processing-e2e-inf-a-50d078-58ffdcc78d-qh9nz 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9 Created pod: payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9 Error creating: pods "payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-50d078: serviceaccount "payload-processing-e2e-inf-a-50d078" not found 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-50d078-58ffdcc78d Error creating: pods "payload-processing-e2e-inf-a-50d078-58ffdcc78d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-50d078: serviceaccount "payload-processing-e2e-inf-a-50d078" not found 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp Add eth0 [10.132.0.224/23] from ovn-kubernetes 34m Normal Started pod/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp Container started 34m Normal Pulled pod/payload-processing-e2e-inf-a-50d078-58ffdcc78d-qh9nz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-inf-a-50d078-58ffdcc78d-qh9nz Container created 34m Normal Created pod/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp Container created 34m Normal AddedInterface pod/payload-processing-e2e-inf-a-50d078-58ffdcc78d-qh9nz Add eth0 [10.132.0.225/23] from ovn-kubernetes 34m Normal Started pod/payload-processing-e2e-inf-a-50d078-58ffdcc78d-qh9nz Container started 34m Normal Pulled pod/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-34e896-6fbfdbbf68 Error creating: pods "payload-processing-e2e-inf-b-34e896-6fbfdbbf68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-34e896: serviceaccount "payload-processing-e2e-inf-b-34e896" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-34e896-6fbfdbbf68 Created pod: payload-processing-e2e-inf-b-34e896-6fbfdbbf68-cr55q 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-34e896 Scaled up replica set payload-processing-e2e-inf-b-34e896-6fbfdbbf68 from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-34e896 Scaled up replica set payload-pre-processing-e2e-inf-b-34e896-68848c7795 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-34e896-68848c7795 Created pod: payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-34e896-68848c7795 Error creating: pods "payload-pre-processing-e2e-inf-b-34e896-68848c7795-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-34e896: serviceaccount "payload-processing-e2e-inf-b-34e896" not found 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg Add eth0 [10.132.0.226/23] from ovn-kubernetes 34m Normal AddedInterface pod/payload-processing-e2e-inf-b-34e896-6fbfdbbf68-cr55q Add eth0 [10.132.0.227/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-inf-b-34e896-6fbfdbbf68-cr55q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg Container created 34m Normal Started pod/payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg Container started 34m Normal Pulled pod/payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-inf-b-34e896-6fbfdbbf68-cr55q Container started 34m Normal Created pod/payload-processing-e2e-inf-b-34e896-6fbfdbbf68-cr55q Container created 34m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default Scaled down replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 5 to 4 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Deleted pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 4; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-vgpvd Stopping container istio-proxy 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-a3204e-58597f8cf5-tczb8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-a3204e-756d475586-r7c4z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Pulled pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Container started 34m Normal Created pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Container created 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4dde48-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4dde48-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning Unhealthy pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Startup probe failed: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Startup probe error: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused... 34m Normal Killing pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Stopping container istio-proxy 34m Warning Unhealthy pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-inf-a-a3204e-openshift-default-57fcb546f7-pfmc5 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/payload-processing-e2e-inf-b-7e233f-b5cfd5c8-8nwls Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr Stopping container payload-pre-processing 34m Normal Pulled pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Pulled pod/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr Container created 34m Normal Started pod/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr Container started 34m Normal Created pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh Container created 34m Normal Started pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh Container started 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-687471-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-687471-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-82e9bf-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-82e9bf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal AddedLabel gateway/e2e-inf-b-e69dee Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-e69dee 34m Normal ScalingReplicaSet deployment/e2e-inf-b-e69dee-openshift-default Scaled up replica set e2e-inf-b-e69dee-openshift-default-869bf96fd9 from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-inf-b-e69dee-openshift-default-869bf96fd9 Created pod: e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 34m Normal AddedInterface pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 Add eth0 [10.133.0.118/23] from ovn-kubernetes 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 6; reason: All metrics below target 34m Warning ProbeError pod/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused... 34m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled down replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 7 to 6 34m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm Stopping container istio-proxy 34m Warning Unhealthy pod/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Deleted pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-7vlzm 33m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba3d7b-openshift-default Scaled down replica set e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 from 4 to 3 33m Normal SuccessfulDelete replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Deleted pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg 33m Normal SuccessfulDelete replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Deleted pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 33m Normal SuccessfulDelete replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Deleted pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 33m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-ptrq5 Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Deleted pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww 33m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-vhvt2 Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default New size: 5; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w6-c4847c-openshift-default Scaled down replica set e2e-worker-w6-c4847c-openshift-default-65bf677b7f from 7 to 5 33m Normal SuccessfulDelete replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Deleted pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn 33m Normal SuccessfulDelete replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Deleted pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 33m Normal Killing pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-hlhww Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 5; reason: All metrics below target 33m Normal SuccessfulDelete replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Deleted pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-2gm4z 33m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-b8tbg Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled down replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 7 to 5 33m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-82tzn Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 5; reason: All metrics below target 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default New size: 3; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled down replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 7 to 5 33m Warning Unhealthy pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-5f492 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-e69dee Scaled up replica set payload-processing-e2e-inf-b-e69dee-f496f5d46 from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-e69dee-f496f5d46 Created pod: payload-processing-e2e-inf-b-e69dee-f496f5d46-npq8n 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff Error creating: pods "payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e69dee: serviceaccount "payload-processing-e2e-inf-b-e69dee" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff Created pod: payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-e69dee Scaled up replica set payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff from 0 to 1 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-e69dee-f496f5d46 Error creating: pods "payload-processing-e2e-inf-b-e69dee-f496f5d46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e69dee: serviceaccount "payload-processing-e2e-inf-b-e69dee" not found 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4dde48-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal AddedInterface pod/payload-processing-e2e-inf-b-e69dee-f496f5d46-npq8n Add eth0 [10.132.0.229/23] from ovn-kubernetes 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4dde48-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 Add eth0 [10.132.0.230/23] from ovn-kubernetes 33m Normal Pulled pod/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Started pod/payload-processing-e2e-inf-b-e69dee-f496f5d46-npq8n Container started 33m Normal Created pod/payload-processing-e2e-inf-b-e69dee-f496f5d46-npq8n Container created 33m Normal Pulled pod/payload-processing-e2e-inf-b-e69dee-f496f5d46-npq8n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 Container created 33m Normal Started pod/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 Container started 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-7e233f-6489d58d4c-2d6lr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal Created pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Container created 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-56d1d9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Pulled pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-56d1d9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Started pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Container started 33m Warning ProbeError pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Readiness probe error: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Readiness probe failed: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Normal Pulled pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Container started 33m Normal Created pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Container created 33m Warning ProbeError pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Startup probe error: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Startup probe failed: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused... 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-687471-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-687471-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning Unhealthy pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-82e9bf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-82e9bf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-29145c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-29145c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-lbk2l Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-7e233f-openshift-default-74c6dbbb65-xpchr Stopping container istio-proxy 33m Normal Started pod/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j Container started 33m Normal Pulled pod/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j Container created 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-29145c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Started pod/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 Container started 33m Normal Pulled pod/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 Container created 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-29145c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-e69dee-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-e69dee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning ProbeError pod/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j Startup probe error: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j Startup probe failed: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-50d078-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-50d078-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-34e896-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 Startup probe failed: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-34e896-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning ProbeError pod/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 Startup probe error: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused... 33m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-n6c2h Stopping container discovery 33m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-n6c2h 33m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 33m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 33m Normal AddedLabel gateway/e2e-inf-b-0e5711 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-0e5711 33m Normal ScalingReplicaSet deployment/e2e-inf-b-0e5711-openshift-default Scaled up replica set e2e-inf-b-0e5711-openshift-default-6f46b7857f from 0 to 1 33m Normal SuccessfulCreate replicaset/e2e-inf-b-0e5711-openshift-default-6f46b7857f Created pod: e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz 33m Normal AddedInterface pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz Add eth0 [10.132.0.231/23] from ovn-kubernetes 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-52f1cb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f Created pod: e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh 33m Normal ScalingReplicaSet deployment/e2e-inf-a-52f1cb-openshift-default Scaled up replica set e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f from 1 to 2 33m Normal Pulled pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 5; reason: All metrics below target 33m Normal AddedInterface pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh Add eth0 [10.132.0.233/23] from ovn-kubernetes 33m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default Scaled down replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 6 to 5 32m Normal SuccessfulDelete replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Deleted pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk 32m Normal Started pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh Container started 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-0e5711-786884fc6d Created pod: payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x 32m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-vzwxk Stopping container istio-proxy 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-56d1d9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default New size: 3; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Deleted pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 32m Normal Created pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh Container created 32m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-0e5711-786884fc6d Error creating: pods "payload-processing-e2e-inf-b-0e5711-786884fc6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0e5711: serviceaccount "payload-processing-e2e-inf-b-0e5711" not found 32m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-6mf46 Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-0e5711 Scaled up replica set payload-processing-e2e-inf-b-0e5711-786884fc6d from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979 Error creating: pods "payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0e5711: serviceaccount "payload-processing-e2e-inf-b-0e5711" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979 Created pod: payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-0e5711 Scaled up replica set payload-pre-processing-e2e-inf-b-0e5711-66d89f5979 from 0 to 1 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-56d1d9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf Add eth0 [10.132.0.234/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf Container started 32m Normal Created pod/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf Container created 32m Normal Pulled pod/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x Container started 32m Normal Created pod/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x Container created 32m Normal Pulled pod/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x Add eth0 [10.132.0.235/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh Startup probe failed: Get "http://10.132.0.233:15021/healthz/ready": dial tcp 10.132.0.233:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh Startup probe error: Get "http://10.132.0.233:15021/healthz/ready": dial tcp 10.132.0.233:15021: connect: connection refused... 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-50d078-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-50d078-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-34e896-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-34e896-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0e5711-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0e5711-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-687471-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-inf-b-687471-openshift-default-79cdb54bc5 Created pod: e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 32m Normal ScalingReplicaSet deployment/e2e-inf-b-687471-openshift-default Scaled up replica set e2e-inf-b-687471-openshift-default-79cdb54bc5 from 1 to 2 32m Normal ScalingReplicaSet deployment/e2e-inf-a-e92dab-openshift-default Scaled up replica set e2e-inf-a-e92dab-openshift-default-55f9bdc949 from 1 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-e92dab-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-inf-a-e92dab-openshift-default-55f9bdc949 Created pod: e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk 32m Normal Started pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Container started 32m Normal Created pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Container created 32m Normal Pulled pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 Add eth0 [10.134.0.117/23] from ovn-kubernetes 32m Normal Created pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 Container created 32m Normal Started pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 Container started 32m Normal AddedInterface pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Add eth0 [10.132.0.237/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning ProbeError pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Startup probe failed: Get "http://10.132.0.237:15021/healthz/ready": dial tcp 10.132.0.237:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Startup probe error: Get "http://10.132.0.237:15021/healthz/ready": dial tcp 10.132.0.237:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused 32m Normal Pulled pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 Container created 32m Normal Started pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 Container started 32m Warning ProbeError pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 31m Normal AddedInterface pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 Add eth0 [10.134.0.119/23] from ovn-kubernetes 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-eacea3-openshift-default Scaled up replica set e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f from 1 to 2 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c Created pod: e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-64f472-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal Created pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 Container created 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-64f472-openshift-default Scaled up replica set e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c from 1 to 2 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f Created pod: e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 31m Normal Started pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 Container started 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-eacea3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal Pulled pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d Add eth0 [10.134.0.120/23] from ovn-kubernetes 31m Normal Started pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d Container started 31m Normal Pulled pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d Container created 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-e69dee-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning Unhealthy pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 Startup probe failed: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 Startup probe error: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-e69dee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning Unhealthy pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused... 31m Normal Started pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz Container started 31m Normal Created pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz Container created 31m Normal Pulled pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning ProbeError pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz Startup probe error: Get "http://10.132.0.231:15021/healthz/ready": dial tcp 10.132.0.231:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz Startup probe failed: Get "http://10.132.0.231:15021/healthz/ready": dial tcp 10.132.0.231:15021: connect: connection refused 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0e5711-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0e5711-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-82e9bf-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal AddedInterface pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Add eth0 [10.132.1.7/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Container created 30m Normal Started pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Container started 30m Normal SuccessfulCreate replicaset/e2e-inf-a-82e9bf-openshift-default-666946c759 Created pod: e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk 30m Normal ScalingReplicaSet deployment/e2e-inf-a-82e9bf-openshift-default Scaled up replica set e2e-inf-a-82e9bf-openshift-default-666946c759 from 1 to 2 30m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-67bf986f99 Deleted pod: maas-default-gateway-openshift-default-67bf986f99-lbh6w 30m Normal Killing pod/maas-default-gateway-openshift-default-67bf986f99-bq26k Stopping container istio-proxy 30m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-67bf986f99 Deleted pod: maas-default-gateway-openshift-default-67bf986f99-bq26k 30m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 30m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-67bf986f99 from 10 to 8 30m Normal Killing pod/maas-default-gateway-openshift-default-67bf986f99-lbh6w Stopping container istio-proxy 30m Normal ScalingReplicaSet deployment/e2e-inf-a-50d078-openshift-default Scaled up replica set e2e-inf-a-50d078-openshift-default-c45b594b7 from 1 to 2 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-50d078-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal SuccessfulCreate replicaset/e2e-inf-a-50d078-openshift-default-c45b594b7 Created pod: e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 30m Normal Created pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Container created 30m Normal Pulled pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Container started 30m Normal AddedInterface pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Add eth0 [10.132.1.8/23] from ovn-kubernetes 30m Warning ProbeError pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Startup probe error: Get "http://10.132.1.7:15021/healthz/ready": dial tcp 10.132.1.7:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Startup probe failed: Get "http://10.132.1.7:15021/healthz/ready": dial tcp 10.132.1.7:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Startup probe failed: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Startup probe error: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused... 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-e69dee-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal SuccessfulCreate replicaset/e2e-inf-b-e69dee-openshift-default-869bf96fd9 Created pod: e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf 30m Normal ScalingReplicaSet deployment/e2e-inf-b-e69dee-openshift-default Scaled up replica set e2e-inf-b-e69dee-openshift-default-869bf96fd9 from 1 to 2 30m Normal AddedInterface pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Add eth0 [10.132.1.10/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Container created 30m Normal Started pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Container started 30m Warning Unhealthy pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Startup probe failed: Get "http://10.132.1.10:15021/healthz/ready": dial tcp 10.132.1.10:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Startup probe error: Get "http://10.132.1.10:15021/healthz/ready": dial tcp 10.132.1.10:15021: connect: connection refused... 29m Normal SuccessfulCreate replicaset/e2e-inf-b-0e5711-openshift-default-6f46b7857f Created pod: e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 29m Normal ScalingReplicaSet deployment/e2e-inf-b-0e5711-openshift-default Scaled up replica set e2e-inf-b-0e5711-openshift-default-6f46b7857f from 1 to 2 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-0e5711-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal AddedInterface pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Add eth0 [10.134.0.124/23] from ovn-kubernetes 29m Normal Created pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Container created 29m Normal Pulled pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Container started 29m Normal Killing pod/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr Stopping container payload-processing 29m Warning Unhealthy pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Startup probe failed: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused 29m Warning ProbeError pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Startup probe error: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused... 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-52f1cb-74c68749f-qx4ps Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-processing-e2e-inf-a-52f1cb-7f8d487558-cqcqr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-e69dee-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Normal ScalingReplicaSet deployment/e2e-inf-b-e69dee-openshift-default Scaled up replica set e2e-inf-b-e69dee-openshift-default-869bf96fd9 from 2 to 3 29m Normal SuccessfulCreate replicaset/e2e-inf-b-e69dee-openshift-default-869bf96fd9 Created pod: e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d 29m Normal AddedInterface pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Add eth0 [10.134.0.125/23] from ovn-kubernetes 29m Normal Started pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Container started 29m Normal Pulled pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Container created 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Deleted pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 3; reason: All metrics below target 29m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-6mxws Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Deleted pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-p84ft 29m Normal SuccessfulDelete replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Deleted pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-rjsw9 29m Normal Killing pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Deleted pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-82b647-openshift-default New size: 1; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w3-82b647-openshift-default Scaled down replica set e2e-worker-w3-82b647-openshift-default-6747d4d57c from 2 to 1 29m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default Scaled down replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 5 to 3 29m Normal SuccessfulDelete replicaset/e2e-worker-w3-82b647-openshift-default-6747d4d57c Deleted pod: e2e-worker-w3-82b647-openshift-default-6747d4d57c-9cjbt 29m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-f67mh Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default New size: 3; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default Scaled down replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 5 to 4 29m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba3d7b-openshift-default Scaled down replica set e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 from 3 to 2 29m Normal Killing pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c4847c-openshift-default New size: 3; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755 Deleted pod: e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba3d7b-openshift-default New size: 2; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w6-c4847c-openshift-default Scaled down replica set e2e-worker-w6-c4847c-openshift-default-65bf677b7f from 5 to 3 29m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 4; reason: All metrics below target 29m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Deleted pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq 29m Normal SuccessfulDelete replicaset/e2e-worker-w6-c4847c-openshift-default-65bf677b7f Deleted pod: e2e-worker-w6-c4847c-openshift-default-65bf677b7f-rwxrm 29m Normal SuccessfulDelete replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Deleted pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-xjvb2 29m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 29m Normal Killing pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-67bf986f99 Deleted pod: maas-default-gateway-openshift-default-67bf986f99-f6dvm 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-67bf986f99 Deleted pod: maas-default-gateway-openshift-default-67bf986f99-4zkk6 29m Normal Killing pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-67bf986f99 from 8 to 6 29m Warning Unhealthy pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Startup probe failed: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused 29m Warning ProbeError pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning ProbeError pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Startup probe error: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-ghs48 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Readiness probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 29m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-gpvwq Readiness probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 29m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Readiness probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 29m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-4zkk6 Readiness probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 29m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Readiness probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-f6dvm Readiness probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Normal Killing pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-2nhqh Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-a-52f1cb-openshift-default-79fc8b8c6f-52t9w Stopping container istio-proxy 28m Normal Killing pod/payload-processing-e2e-inf-b-687471-74d85f5bdd-qw568 Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 Stopping container payload-pre-processing 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-29145c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f Created pod: e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-29145c-openshift-default Scaled up replica set e2e-shared-b-w5-29145c-openshift-default-56b4496f7f from 1 to 2 28m Normal AddedInterface pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs Add eth0 [10.132.1.24/23] from ovn-kubernetes 28m Normal Started pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs Container started 28m Normal Created pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs Container created 28m Normal Pulled pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs Startup probe error: Get "http://10.132.1.24:15021/healthz/ready": dial tcp 10.132.1.24:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs Startup probe failed: Get "http://10.132.1.24:15021/healthz/ready": dial tcp 10.132.1.24:15021: connect: connection refused 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc Stopping container payload-pre-processing 28m Warning Unhealthy pod/payload-processing-e2e-inf-a-4dde48-8f44b4878-66jr2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-4dde48-745c9456d7-k5tq9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-687471-57fb85bd67-twh2w Readiness probe failed: timeout: failed to connect service "10.133.0.113:50052" within 1s: context deadline exceeded 28m Warning Unhealthy pod/payload-processing-e2e-inf-a-e92dab-76bcb94f8b-hs4z4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-fdg72 Stopping container istio-proxy 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-e92dab-dbcfdfcc5-2nsjc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning ProbeError pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-e92dab-openshift-default-55f9bdc949-w5vgk Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-eacea3-openshift-default Scaled up replica set e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f from 2 to 3 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-eacea3-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f Created pod: e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq 28m Normal Killing pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-ggsx8 Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-687471-openshift-default-79cdb54bc5-2pskh Stopping container istio-proxy 28m Normal Pulled pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq Container created 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc Created pod: e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb 28m Normal AddedLabel gateway/e2e-shared-a-w3-e4f8a6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-e4f8a6 28m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-v5jpm 28m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Readiness probe failed: Get "http://10.133.0.76:8080/ready": dial tcp 10.133.0.76:8080: connect: connection refused 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-e4f8a6-openshift-default Scaled up replica set e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc from 0 to 1 28m Normal AddedInterface pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq Add eth0 [10.133.0.132/23] from ovn-kubernetes 28m Normal Started pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq Container started 28m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Stopping container discovery 28m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-v5jpm Readiness probe error: Get "http://10.133.0.76:8080/ready": dial tcp 10.133.0.76:8080: connect: connection refused... 28m Normal AddedInterface pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Add eth0 [10.133.0.133/23] from ovn-kubernetes 28m Normal Killing pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Stopping container istio-proxy 28m Warning ProbeError pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-inf-a-4dde48-openshift-default-7599b7cfcc-wg7cr Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal Killing pod/payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b Stopping container payload-processing 28m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998 Error creating: pods "payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-e4f8a6: serviceaccount "payload-processing-e2e-shared-a-w3-e4f8a6" not found 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 Stopping container payload-pre-processing 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-e4f8a6 Scaled up replica set payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998 from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c Error creating: pods "payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-e4f8a6: serviceaccount "payload-processing-e2e-shared-a-w3-e4f8a6" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c Created pod: payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-e4f8a6 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998 Created pod: payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh 28m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Killing pod/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p Stopping container payload-processing 28m Warning ProbeError pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq Startup probe error: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused... 28m Normal Pulled pod/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh Add eth0 [10.133.0.134/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh Container started 28m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm Add eth0 [10.133.0.135/23] from ovn-kubernetes 28m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm Container started 28m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm Container created 28m Warning Unhealthy pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq Startup probe failed: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused 28m Normal Created pod/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh Container created 27m Normal Killing pod/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht Stopping container payload-pre-processing 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-d827b7-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 27m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm Stopping container istio-proxy 27m Normal SuccessfulDelete replicaset/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c Deleted pod: e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-ws8gm 27m Normal ScalingReplicaSet deployment/e2e-worker-w4-d827b7-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c from 3 to 2 27m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-67bf986f99 Deleted pod: maas-default-gateway-openshift-default-67bf986f99-9x6g8 27m Normal Killing pod/maas-default-gateway-openshift-default-67bf986f99-9x6g8 Stopping container istio-proxy 27m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 27m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-67bf986f99 from 6 to 5 27m Normal Killing pod/payload-processing-e2e-inf-b-34e896-6fbfdbbf68-cr55q Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg Stopping container payload-pre-processing 27m Warning Unhealthy pod/payload-processing-e2e-inf-a-82e9bf-7f866bf6dd-29g4p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-82e9bf-75b57bb5cf-qjsht Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-56d1d9-7bf84d88bc-kj8k9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-inf-b-56d1d9-59985c8c84-kj79b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/payload-processing-e2e-inf-a-50d078-58ffdcc78d-qh9nz Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp Stopping container payload-pre-processing 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-34e896-68848c7795-v8xfg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-inf-a-82e9bf-openshift-default-666946c759-hldpr Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Stopping container istio-proxy 27m Warning ProbeError pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-inf-a-82e9bf-openshift-default-666946c759-4q4sk Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning Unhealthy pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-inf-b-56d1d9-openshift-default-5f88488bc9-85f26 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-a-50d078-openshift-default-c45b594b7-wpd9j Stopping container istio-proxy 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-0e5711-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 27m Normal SuccessfulCreate replicaset/e2e-inf-b-0e5711-openshift-default-6f46b7857f Created pod: e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh 27m Normal ScalingReplicaSet deployment/e2e-inf-b-0e5711-openshift-default Scaled up replica set e2e-inf-b-0e5711-openshift-default-6f46b7857f from 2 to 3 27m Warning ProbeError pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Started pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Container started 27m Warning Unhealthy pod/e2e-inf-a-50d078-openshift-default-c45b594b7-bn2p9 Readiness probe failed: HTTP probe failed with statuscode: 503 27m Normal Killing pod/e2e-inf-b-34e896-openshift-default-6c7475f655-grr54 Stopping container istio-proxy 27m Normal Created pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Container created 27m Normal Pulled pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Add eth0 [10.133.0.137/23] from ovn-kubernetes 27m Normal Killing pod/payload-processing-e2e-inf-b-e69dee-f496f5d46-npq8n Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 Stopping container payload-pre-processing 27m Warning ProbeError pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Startup probe error: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Startup probe failed: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-50d078-7cb9f878f9-wc6qp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-69dc90-openshift-default-df55b565f Created pod: e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-69dc90-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf Stopping container payload-pre-processing 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-69dc90-openshift-default Scaled up replica set e2e-shared-b-w0-69dc90-openshift-default-df55b565f from 1 to 2 27m Normal Killing pod/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x Stopping container payload-processing 27m Normal Pulled pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw Container started 27m Normal Created pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw Container created 27m Normal AddedInterface pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw Add eth0 [10.133.0.138/23] from ovn-kubernetes 27m Warning ProbeError pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw Startup probe error: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused... 27m Warning ProbeError pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Readiness probe error: Get "http://10.133.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 27m Warning Unhealthy pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw Startup probe failed: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Readiness probe failed: Get "http://10.133.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 27m Normal Pulled pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Container started 26m Normal Created pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Container created 26m Normal SuccessfulDelete replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Deleted pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 26m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default New size: 3; reason: All metrics below target 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-e69dee-56b8f7d8ff-czkm6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning ProbeError pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Startup probe error: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Startup probe failed: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused 26m Warning Unhealthy pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Readiness probe failed: HTTP probe failed with statuscode: 503 26m Normal Killing pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Stopping container istio-proxy 26m Warning ProbeError pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal Killing pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-mjskf Stopping container istio-proxy 26m Normal Killing pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-pfvt8 Stopping container istio-proxy 26m Warning Unhealthy pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Readiness probe error: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c Created pod: e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 26m Normal AddedLabel gateway/e2e-shared-a-w4-e2d6d3 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-e2d6d3 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e2d6d3-openshift-default Scaled up replica set e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c from 0 to 1 26m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-zmlk8 Readiness probe failed: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning ProbeError pod/e2e-inf-b-e69dee-openshift-default-869bf96fd9-tk48d Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal AddedLabel gateway/e2e-shared-b-w3-886adb Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-886adb 26m Normal AddedInterface pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 Add eth0 [10.132.1.38/23] from ovn-kubernetes 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-886adb-openshift-default Scaled up replica set e2e-shared-b-w3-886adb-openshift-default-64589cdbfc from 0 to 1 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 26m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Add eth0 [10.132.1.39/23] from ovn-kubernetes 26m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d Error creating: pods "payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-e2d6d3: serviceaccount "payload-processing-e2e-shared-a-w4-e2d6d3" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff Created pod: payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-mrztt 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-e2d6d3 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d Created pod: payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-9z2kk 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff Error creating: pods "payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-e2d6d3: serviceaccount "payload-processing-e2e-shared-a-w4-e2d6d3" not found 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-e2d6d3 Scaled up replica set payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d from 0 to 1 26m Normal Killing pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Stopping container istio-proxy 26m Normal Killing pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-n2lkz Stopping container istio-proxy 26m Normal Killing pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Stopping container istio-proxy 26m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-mrztt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-mrztt Add eth0 [10.132.1.40/23] from ovn-kubernetes 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f Created pod: e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 26m Normal Started pod/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-9z2kk Container started 26m Normal Created pod/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-9z2kk Container created 26m Normal AddedLabel gateway/e2e-shared-a-w6-d81f0a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-d81f0a 26m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-9z2kk Add eth0 [10.132.1.42/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-mrztt Container started 26m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-mrztt Container created 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-d81f0a-openshift-default Scaled up replica set e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f from 0 to 1 26m Normal Pulled pod/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-9z2kk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Warning Unhealthy pod/payload-processing-e2e-inf-b-0e5711-786884fc6d-hzk2x Readiness probe failed: timeout: failed to connect service "10.132.0.235:50052" within 1s: context deadline exceeded 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-0e5711-66d89f5979-kd7mf Readiness probe failed: timeout: failed to connect service "10.132.0.234:50052" within 1s: context deadline exceeded 26m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978 Error creating: pods "payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-886adb: serviceaccount "payload-processing-e2e-shared-b-w3-886adb" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c Created pod: payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-886adb Scaled up replica set payload-processing-e2e-shared-b-w3-886adb-5bdf49f978 from 0 to 1 26m Normal AddedInterface pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 Add eth0 [10.132.1.43/23] from ovn-kubernetes 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-886adb Scaled up replica set payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978 Created pod: payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c Error creating: pods "payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-886adb: serviceaccount "payload-processing-e2e-shared-b-w3-886adb" not found 26m Warning Unhealthy pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 Container created 26m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Pulled pod/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m Add eth0 [10.132.1.45/23] from ovn-kubernetes 26m Warning ProbeError pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-m6kh8 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal Started pod/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m Container started 26m Normal Created pod/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m Container created 26m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 Add eth0 [10.132.1.44/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 Container started 26m Warning ProbeError pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-inf-b-0e5711-openshift-default-6f46b7857f-r4gxh Readiness probe failed: HTTP probe failed with statuscode: 503 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-d81f0a Scaled up replica set payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-d81f0a Scaled up replica set payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76 from 0 to 1 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-e4f8a6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-e4f8a6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76 Error creating: pods "payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-d81f0a: serviceaccount "payload-processing-e2e-shared-a-w6-d81f0a" not found 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-d81f0a: serviceaccount "payload-processing-e2e-shared-a-w6-d81f0a" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76 Created pod: payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99 Created pod: payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt 26m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt Add eth0 [10.132.1.46/23] from ovn-kubernetes 26m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj Add eth0 [10.132.1.47/23] from ovn-kubernetes 26m Normal Pulled pod/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj Container created 26m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt Container created 26m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt Container started 26m Normal Started pod/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj Container started 26m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-67bf986f99 from 5 to 4 26m Normal Killing pod/maas-default-gateway-openshift-default-67bf986f99-8ntx4 Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 26m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-67bf986f99 Deleted pod: maas-default-gateway-openshift-default-67bf986f99-8ntx4 26m Normal Started pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 Container started 26m Normal Created pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 Container created 26m Normal Pulled pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 Add eth0 [10.133.0.141/23] from ovn-kubernetes 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9 Created pod: e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-18a189-openshift-default Scaled up replica set e2e-shared-a-w2-18a189-openshift-default-76698f9dc9 from 1 to 2 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-18a189-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 26m Warning Unhealthy pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f Created pod: e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-29145c-openshift-default Scaled up replica set e2e-shared-b-w5-29145c-openshift-default-56b4496f7f from 2 to 3 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-29145c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-d32079-openshift-default-7b55d86997 Created pod: e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp 26m Normal AddedInterface pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz Add eth0 [10.133.0.142/23] from ovn-kubernetes 26m Normal Started pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz Container started 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-d32079-openshift-default Scaled up replica set e2e-ipp-a-d32079-openshift-default-7b55d86997 from 0 to 1 26m Normal Created pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz Container created 26m Normal Pulled pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedLabel gateway/e2e-ipp-a-d32079 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-d32079 25m Normal AddedInterface pod/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp Add eth0 [10.132.1.53/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-262230-openshift-default Scaled up replica set e2e-shared-a-w1-262230-openshift-default-9d45d8bbc from 1 to 2 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc Created pod: e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-262230-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal Created pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 Container created 25m Normal Started pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 Container started 25m Normal Pulled pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 Add eth0 [10.132.1.54/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-d32079 Scaled up replica set payload-processing-e2e-ipp-a-d32079-58d9d7bc8b from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5 Created pod: payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-fpvjm 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b Error creating: pods "payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d32079: serviceaccount "payload-processing-e2e-ipp-a-d32079" not found 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5 Error creating: pods "payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d32079: serviceaccount "payload-processing-e2e-ipp-a-d32079" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-d32079 Scaled up replica set payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b Created pod: payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-wzfkc 25m Warning ProbeError pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz Startup probe error: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused... 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-fpvjm Add eth0 [10.132.1.55/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-fpvjm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz Startup probe failed: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-wzfkc Add eth0 [10.132.1.56/23] from ovn-kubernetes 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-fpvjm Container created 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-fpvjm Container started 25m Normal Pulled pod/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-wzfkc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-wzfkc Container created 25m Normal Started pod/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-wzfkc Container started 25m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Readiness probe error: Get "http://10.132.1.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Readiness probe failed: Get "http://10.132.1.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Warning ProbeError pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 Startup probe error: Get "http://10.132.1.54:15021/healthz/ready": dial tcp 10.132.1.54:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 Startup probe failed: Get "http://10.132.1.54:15021/healthz/ready": dial tcp 10.132.1.54:15021: connect: connection refused 25m Normal AddedLabel gateway/e2e-shared-b-w4-999ff9 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-999ff9 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-999ff9-openshift-default Scaled up replica set e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c Created pod: e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 25m Normal AddedInterface pod/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 Add eth0 [10.134.0.130/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c Created pod: e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-e2d6d3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-e2d6d3-openshift-default Scaled up replica set e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c from 1 to 2 25m Normal AddedInterface pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp Add eth0 [10.133.0.143/23] from ovn-kubernetes 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f Error creating: pods "payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-999ff9: serviceaccount "payload-processing-e2e-shared-b-w4-999ff9" not found 25m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd Error creating: pods "payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-999ff9: serviceaccount "payload-processing-e2e-shared-b-w4-999ff9" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-999ff9 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f Created pod: payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-cg2gj 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd Created pod: payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-rmjfn 25m Normal Started pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp Container started 25m Normal Created pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp Container created 25m Normal Pulled pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-999ff9 Scaled up replica set payload-processing-e2e-shared-b-w4-999ff9-757d5774cd from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t 25m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Add eth0 [10.132.1.60/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-886adb-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s Container created 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-886adb-openshift-default Scaled up replica set e2e-shared-b-w3-886adb-openshift-default-64589cdbfc from 1 to 4 25m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-rmjfn Add eth0 [10.132.1.58/23] from ovn-kubernetes 25m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s Add eth0 [10.134.0.131/23] from ovn-kubernetes 25m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl Add eth0 [10.133.0.144/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl Container created 25m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl Container started 25m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-cg2gj Container started 25m Normal Started pod/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-rmjfn Container started 25m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-cg2gj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-cg2gj Add eth0 [10.132.1.59/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl 25m Normal Created pod/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-rmjfn Container created 25m Normal Pulled pod/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-rmjfn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s Container started 25m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-cg2gj Container created 25m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Container started 25m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Container created 25m Normal Started pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 Container started 25m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Container created 25m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Pulled pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Container started 25m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Container started 25m Normal Created pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 Container created 25m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Container created 25m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e92648-openshift-default Scaled up replica set e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v 25m Normal AddedLabel gateway/e2e-shared-b-w6-e92648 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-e92648 25m Warning ProbeError pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 25m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-2be695-openshift-default Scaled up replica set e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 from 1 to 4 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp 25m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Add eth0 [10.132.1.61/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 Container started 25m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp Container started 25m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 Add eth0 [10.133.0.146/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp Container created 25m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 Container created 25m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp Add eth0 [10.134.0.132/23] from ovn-kubernetes 25m Warning ProbeError pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 Startup probe error: Get "http://10.132.1.38:15021/healthz/ready": dial tcp 10.132.1.38:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Startup probe failed: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Startup probe failed: Get "http://10.132.1.39:15021/healthz/ready": dial tcp 10.132.1.39:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Startup probe error: Get "http://10.132.1.39:15021/healthz/ready": dial tcp 10.132.1.39:15021: connect: connection refused... 25m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Startup probe error: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 Startup probe failed: Get "http://10.132.1.38:15021/healthz/ready": dial tcp 10.132.1.38:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Startup probe error: Get "http://10.132.1.60:15021/healthz/ready": dial tcp 10.132.1.60:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Startup probe failed: Get "http://10.132.1.60:15021/healthz/ready": dial tcp 10.132.1.60:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 Startup probe failed: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused 25m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Add eth0 [10.132.1.62/23] from ovn-kubernetes 25m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Container started 25m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Container created 25m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 Startup probe error: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused... 25m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp Startup probe error: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp Startup probe failed: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused 25m Normal AddedLabel gateway/e2e-ipp-a-4be164 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-4be164 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-4be164-openshift-default-7446b5bdf9 Created pod: e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-4be164-openshift-default Scaled up replica set e2e-ipp-a-4be164-openshift-default-7446b5bdf9 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-e92648: serviceaccount "payload-processing-e2e-shared-b-w6-e92648" not found 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-e92648 Scaled up replica set payload-processing-e2e-shared-b-w6-e92648-57477cbddb from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-e92648 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99 Created pod: payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt 25m Normal AddedInterface pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Add eth0 [10.132.1.63/23] from ovn-kubernetes 25m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-e92648-57477cbddb Error creating: pods "payload-processing-e2e-shared-b-w6-e92648-57477cbddb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-e92648: serviceaccount "payload-processing-e2e-shared-b-w6-e92648" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-e92648-57477cbddb Created pod: payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh 25m Normal Started pod/payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh Container started 25m Normal Pulled pod/payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt Container created 25m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Startup probe failed: Get "http://10.132.1.62:15021/healthz/ready": dial tcp 10.132.1.62:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Startup probe error: Get "http://10.132.1.62:15021/healthz/ready": dial tcp 10.132.1.62:15021: connect: connection refused... 25m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt Container started 25m Normal Created pod/payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh Container created 25m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh Add eth0 [10.133.0.148/23] from ovn-kubernetes 25m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt Add eth0 [10.133.0.147/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-4be164 Scaled up replica set payload-processing-e2e-ipp-a-4be164-6d67f7db44 from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-4be164-6d67f7db44 Error creating: pods "payload-processing-e2e-ipp-a-4be164-6d67f7db44-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4be164: serviceaccount "payload-processing-e2e-ipp-a-4be164" not found 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-4be164-5676d98555 Error creating: pods "payload-pre-processing-e2e-ipp-a-4be164-5676d98555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4be164: serviceaccount "payload-processing-e2e-ipp-a-4be164" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-4be164-5676d98555 Created pod: payload-pre-processing-e2e-ipp-a-4be164-5676d98555-tgtpr 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-4be164-6d67f7db44 Created pod: payload-processing-e2e-ipp-a-4be164-6d67f7db44-ll6wk 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-4be164 Scaled up replica set payload-pre-processing-e2e-ipp-a-4be164-5676d98555 from 0 to 1 25m Normal Started pod/payload-processing-e2e-ipp-a-4be164-6d67f7db44-ll6wk Container started 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-4be164-6d67f7db44-ll6wk Add eth0 [10.132.1.64/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-4be164-5676d98555-tgtpr Container started 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-4be164-5676d98555-tgtpr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Pulled pod/payload-processing-e2e-ipp-a-4be164-6d67f7db44-ll6wk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-4be164-5676d98555-tgtpr Container created 25m Normal Created pod/payload-processing-e2e-ipp-a-4be164-6d67f7db44-ll6wk Container created 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-4be164-5676d98555-tgtpr Add eth0 [10.133.0.149/23] from ovn-kubernetes 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-e2d6d3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-e2d6d3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-886adb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-886adb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz Add eth0 [10.132.1.65/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-886adb-openshift-default Scaled up replica set e2e-shared-b-w3-886adb-openshift-default-64589cdbfc from 4 to 8 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-886adb-openshift-default New size: 8; reason: All metrics below target 24m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx Container created 24m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx Add eth0 [10.133.0.152/23] from ovn-kubernetes 24m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp Add eth0 [10.134.0.134/23] from ovn-kubernetes 24m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx Container started 24m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s Add eth0 [10.134.0.135/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s Container created 24m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp Container started 24m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx 24m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp Container created 24m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s Container started 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s 24m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz Container created 24m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz Container started 24m Normal Pulled pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 Container created 24m Normal Started pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 Container started 24m Normal Started pod/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp Container started 24m Normal Created pod/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp Container created 24m Normal Pulled pod/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-d32079-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-2be695-openshift-default Scaled up replica set e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 from 4 to 8 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default New size: 8; reason: 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-d32079-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp Startup probe failed: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused 24m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp Startup probe error: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused... 24m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s Startup probe error: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 24m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx Container created 24m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl Add eth0 [10.134.0.136/23] from ovn-kubernetes 24m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl Container created 24m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 24m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl Container started 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-d81f0a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s Startup probe failed: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-d81f0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx Add eth0 [10.133.0.155/23] from ovn-kubernetes 24m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx Container started 24m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v Container created 24m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v Container started 24m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v Add eth0 [10.133.0.154/23] from ovn-kubernetes 24m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz Startup probe error: Get "http://10.132.1.65:15021/healthz/ready": dial tcp 10.132.1.65:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz Startup probe failed: Get "http://10.132.1.65:15021/healthz/ready": dial tcp 10.132.1.65:15021: connect: connection refused 24m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Container created 24m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Add eth0 [10.132.1.66/23] from ovn-kubernetes 24m Warning ProbeError pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 Startup probe error: Get "http://10.132.1.43:15021/healthz/ready": dial tcp 10.132.1.43:15021: connect: connection refused... 24m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Container started 24m Warning Unhealthy pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 Startup probe failed: Get "http://10.132.1.43:15021/healthz/ready": dial tcp 10.132.1.43:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp Startup probe error: Get "http://10.132.1.53:15021/healthz/ready": dial tcp 10.132.1.53:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp Startup probe failed: Get "http://10.132.1.53:15021/healthz/ready": dial tcp 10.132.1.53:15021: connect: connection refused 24m Normal AddedLabel gateway/e2e-ipp-b-b2a05c Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-b2a05c 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-b2a05c-openshift-default Scaled up replica set e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 from 0 to 1 24m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl Startup probe failed: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl Startup probe error: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused... 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-d32079-6c67d697c5-fpvjm Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-a-d32079-58d9d7bc8b-wzfkc Stopping container payload-processing 24m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx Startup probe failed: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused 24m Normal AddedInterface pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Add eth0 [10.132.1.67/23] from ovn-kubernetes 24m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 24m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx Startup probe error: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused... 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-886adb-openshift-default Scaled up replica set e2e-shared-b-w3-886adb-openshift-default-64589cdbfc from 8 to 10 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc (combined from similar events): Created pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk 24m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq Container created 24m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq Add eth0 [10.133.0.156/23] from ovn-kubernetes 24m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq Container started 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-886adb-openshift-default New size: 10; reason: All metrics below target 24m Normal Created pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk Container created 24m Normal AddedInterface pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk Add eth0 [10.132.1.68/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk Container started 24m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Startup probe error: Get "http://10.132.1.66:15021/healthz/ready": dial tcp 10.132.1.66:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Startup probe failed: Get "http://10.132.1.66:15021/healthz/ready": dial tcp 10.132.1.66:15021: connect: connection refused 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47 Error creating: pods "payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-b2a05c: serviceaccount "payload-processing-e2e-ipp-b-b2a05c" not found 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd Error creating: pods "payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-b2a05c: serviceaccount "payload-processing-e2e-ipp-b-b2a05c" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-b2a05c Scaled up replica set payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-b2a05c Scaled up replica set payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47 from 0 to 1 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-9vx7m Add eth0 [10.132.1.69/23] from ovn-kubernetes 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47 Created pod: payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-9vx7m 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd Created pod: payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-vkxqt 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-9vx7m Container created 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-vkxqt Add eth0 [10.132.1.70/23] from ovn-kubernetes 24m Normal Started pod/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-vkxqt Container started 24m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq Startup probe error: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused... 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-9vx7m Container started 24m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq Startup probe failed: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused 24m Normal Pulled pod/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-vkxqt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-9vx7m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-vkxqt Container created 24m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk Startup probe failed: Get "http://10.132.1.68:15021/healthz/ready": dial tcp 10.132.1.68:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk Startup probe error: Get "http://10.132.1.68:15021/healthz/ready": dial tcp 10.132.1.68:15021: connect: connection refused... 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-d32079-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-d32079-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-d81f0a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-d81f0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal Pulled pod/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 Container started 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f Error creating: pods "payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d32079: serviceaccount "payload-processing-e2e-ipp-a-d32079" not found 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-d32079 Scaled up replica set payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f from 0 to 1 24m Normal Created pod/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 Container created 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-d32079-65bd949f84 Created pod: payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-d32079-65bd949f84 Error creating: pods "payload-processing-e2e-ipp-a-d32079-65bd949f84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d32079: serviceaccount "payload-processing-e2e-ipp-a-d32079" not found 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f Created pod: payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-d32079 Scaled up replica set payload-processing-e2e-ipp-a-d32079-65bd949f84 from 0 to 1 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw Add eth0 [10.132.1.73/23] from ovn-kubernetes 24m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 24m Normal Pulling pod/payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs Add eth0 [10.134.0.138/23] from ovn-kubernetes 24m Normal Pulled pod/payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.88s (1.88s including waiting). Image size: 166612641 bytes. 24m Normal Started pod/payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs Container started 24m Normal Created pod/payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw Container started 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.831s (2.831s including waiting). Image size: 166612641 bytes. 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw Container created 23m Warning Unhealthy pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Readiness probe failed: Get "http://10.132.1.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Warning ProbeError pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Readiness probe error: Get "http://10.132.1.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Normal Created pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Container created 23m Normal Started pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Container started 23m Warning ProbeError pod/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 Startup probe error: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 Startup probe failed: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused 23m Normal Pulled pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-999ff9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-999ff9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 Created pod: e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j 23m Warning Unhealthy pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Startup probe failed: Get "http://10.132.1.63:15021/healthz/ready": dial tcp 10.132.1.63:15021: connect: connection refused 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-4be164-5676d98555-tgtpr Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-4be164-6d67f7db44-ll6wk Stopping container payload-processing 23m Warning ProbeError pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Startup probe error: Get "http://10.132.1.63:15021/healthz/ready": dial tcp 10.132.1.63:15021: connect: connection refused... 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-add1bb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Readiness probe failed: Get "http://10.132.1.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-add1bb-openshift-default Scaled up replica set e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 from 1 to 2 23m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Readiness probe error: Get "http://10.132.1.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Normal ScalingReplicaSet deployment/e2e-ipp-b-7f0263-openshift-default Scaled up replica set e2e-ipp-b-7f0263-openshift-default-6c6d84745c from 0 to 1 23m Normal AddedLabel gateway/e2e-ipp-b-7f0263 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-7f0263 23m Normal SuccessfulCreate replicaset/e2e-ipp-b-7f0263-openshift-default-6c6d84745c Created pod: e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp 23m Normal AddedInterface pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j Add eth0 [10.132.1.76/23] from ovn-kubernetes 23m Normal Pulled pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j Container created 23m Normal Started pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j Container started 23m Normal AddedInterface pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Add eth0 [10.132.1.77/23] from ovn-kubernetes 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-e92648-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx 23m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e92648-openshift-default Scaled up replica set e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 from 1 to 4 23m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 Add eth0 [10.133.0.159/23] from ovn-kubernetes 23m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb Container created 23m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 Container started 23m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 Container created 23m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb Container started 23m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx Add eth0 [10.134.0.140/23] from ovn-kubernetes 23m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx Container created 23m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx Container started 23m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb Add eth0 [10.133.0.158/23] from ovn-kubernetes 23m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-2be695-openshift-default Scaled up replica set e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 from 8 to 9 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 23m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Container started 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd Created pod: payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-k7kp5 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-7f0263 Scaled up replica set payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c Created pod: payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-4s82z 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c Error creating: pods "payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7f0263: serviceaccount "payload-processing-e2e-ipp-b-7f0263" not found 23m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-7f0263 Scaled up replica set payload-processing-e2e-ipp-b-7f0263-668cfc5fcd from 0 to 1 23m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt Container started 23m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt Container created 23m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Container created 23m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt Add eth0 [10.134.0.141/23] from ovn-kubernetes 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd Error creating: pods "payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7f0263: serviceaccount "payload-processing-e2e-ipp-b-7f0263" not found 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-4s82z Add eth0 [10.132.1.78/23] from ovn-kubernetes 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-4s82z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-262230-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 23m Warning ProbeError pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j Startup probe error: Get "http://10.132.1.76:15021/healthz/ready": dial tcp 10.132.1.76:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j Startup probe failed: Get "http://10.132.1.76:15021/healthz/ready": dial tcp 10.132.1.76:15021: connect: connection refused 23m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-262230-openshift-default Scaled up replica set e2e-shared-a-w1-262230-openshift-default-9d45d8bbc from 2 to 3 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc Created pod: e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 23m Normal AddedInterface pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 Add eth0 [10.134.0.142/23] from ovn-kubernetes 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-4s82z Container created 23m Normal Created pod/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-k7kp5 Container created 23m Normal Started pod/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-k7kp5 Container started 23m Normal Pulled pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 Container created 23m Normal Pulled pod/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-k7kp5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 Container started 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-k7kp5 Add eth0 [10.132.1.79/23] from ovn-kubernetes 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-4s82z Container started 23m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx Startup probe failed: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 Startup probe error: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 Startup probe failed: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx Startup probe error: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb Startup probe failed: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb Startup probe error: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused... 23m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused 23m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Startup probe failed: Get "http://10.132.1.61:15021/healthz/ready": dial tcp 10.132.1.61:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Startup probe error: Get "http://10.132.1.61:15021/healthz/ready": dial tcp 10.132.1.61:15021: connect: connection refused... 23m Warning ProbeError pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 Startup probe error: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 Startup probe failed: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-e92648-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-e92648-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-999ff9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-999ff9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal AddedLabel gateway/e2e-ipp-a-73831d Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-73831d 23m Normal ScalingReplicaSet deployment/e2e-ipp-a-73831d-openshift-default Scaled up replica set e2e-ipp-a-73831d-openshift-default-d94dc898c from 0 to 1 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-73831d-openshift-default-d94dc898c Created pod: e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w 23m Normal AddedInterface pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Add eth0 [10.132.1.80/23] from ovn-kubernetes 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-4be164-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-e92648-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal AddedLabel gateway/e2e-ipp-a-d3ed60 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-d3ed60 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-e92648-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-4be164-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal ScalingReplicaSet deployment/e2e-ipp-a-d3ed60-openshift-default Scaled up replica set e2e-ipp-a-d3ed60-openshift-default-7888cd9c47 from 0 to 1 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47 Created pod: e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj 23m Normal AddedInterface pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Add eth0 [10.132.1.81/23] from ovn-kubernetes 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f Error creating: pods "payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73831d: serviceaccount "payload-processing-e2e-ipp-a-73831d" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f Created pod: payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-tcrmb 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-73831d Scaled up replica set payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f from 0 to 1 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-73831d-75bf4f98d Error creating: pods "payload-processing-e2e-ipp-a-73831d-75bf4f98d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73831d: serviceaccount "payload-processing-e2e-ipp-a-73831d" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-73831d-75bf4f98d Created pod: payload-processing-e2e-ipp-a-73831d-75bf4f98d-2lkh5 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-73831d Scaled up replica set payload-processing-e2e-ipp-a-73831d-75bf4f98d from 0 to 1 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-tcrmb Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-tcrmb Container started 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-tcrmb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-tcrmb Add eth0 [10.133.0.162/23] from ovn-kubernetes 23m Normal Pulled pod/payload-processing-e2e-ipp-a-73831d-75bf4f98d-2lkh5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-73831d-75bf4f98d-2lkh5 Add eth0 [10.132.1.82/23] from ovn-kubernetes 23m Normal Created pod/payload-processing-e2e-ipp-a-73831d-75bf4f98d-2lkh5 Container created 23m Normal Started pod/payload-processing-e2e-ipp-a-73831d-75bf4f98d-2lkh5 Container started 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d Error creating: pods "payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d3ed60: serviceaccount "payload-processing-e2e-ipp-a-d3ed60" not found 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-d3ed60 Scaled up replica set payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d from 0 to 1 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff Error creating: pods "payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-d3ed60: serviceaccount "payload-processing-e2e-ipp-a-d3ed60" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff Created pod: payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-d3ed60 Scaled up replica set payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d Created pod: payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-dbpp4 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-4be164 Scaled up replica set payload-processing-e2e-ipp-a-4be164-646f4df6bb from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-4be164-646f4df6bb Created pod: payload-processing-e2e-ipp-a-4be164-646f4df6bb-8bxsj 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-4be164 Scaled up replica set payload-pre-processing-e2e-ipp-a-4be164-7675ddb75 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75 Created pod: payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75 Error creating: pods "payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4be164: serviceaccount "payload-processing-e2e-ipp-a-4be164" not found 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-4be164-646f4df6bb Error creating: pods "payload-processing-e2e-ipp-a-4be164-646f4df6bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4be164: serviceaccount "payload-processing-e2e-ipp-a-4be164" not found 23m Normal Created pod/payload-processing-e2e-ipp-a-4be164-646f4df6bb-8bxsj Container created 23m Normal Started pod/payload-processing-e2e-ipp-a-4be164-646f4df6bb-8bxsj Container started 23m Normal Pulled pod/payload-processing-e2e-ipp-a-4be164-646f4df6bb-8bxsj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-4be164-646f4df6bb-8bxsj Add eth0 [10.134.0.143/23] from ovn-kubernetes 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct Add eth0 [10.132.1.83/23] from ovn-kubernetes 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt Readiness probe failed: timeout: failed to connect service "10.132.0.221:50052" within 1s: context deadline exceeded 23m Warning Unhealthy pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Readiness probe failed: Get "http://10.132.0.169:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Normal Pulled pod/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-dbpp4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-dbpp4 Container created 23m Normal Started pod/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-dbpp4 Container started 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm Readiness probe failed: timeout: failed to connect service "10.132.0.188:50052" within 1s: context deadline exceeded 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct Container created 23m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc Readiness probe failed: timeout: failed to connect service "10.132.0.59:50052" within 1s: context deadline exceeded 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-dbpp4 Add eth0 [10.132.1.84/23] from ovn-kubernetes 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h Readiness probe failed: timeout: failed to connect service "10.132.0.171:50052" within 1s: context deadline exceeded 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct Container started 23m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf Readiness probe failed: timeout: failed to connect service "10.132.0.187:50052" within 1s: context deadline exceeded 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Warning ProbeError pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Readiness probe error: Get "http://10.132.0.169:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph Container started 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph Add eth0 [10.132.1.85/23] from ovn-kubernetes 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph Container created 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-e92648-openshift-default New size: 8; reason: All metrics below target 23m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e92648-openshift-default Scaled up replica set e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 from 4 to 8 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 23m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c Container created 23m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c Add eth0 [10.134.0.144/23] from ovn-kubernetes 23m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 Container started 23m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 Container created 23m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c Container started 23m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 Add eth0 [10.133.0.163/23] from ovn-kubernetes 23m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g Add eth0 [10.132.1.87/23] from ovn-kubernetes 23m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Killing pod/payload-processing-e2e-ipp-b-b2a05c-6d9b9f68dd-vkxqt Stopping container payload-processing 23m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g Container started 23m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g Container created 23m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 Container started 23m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 Container created 23m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 Add eth0 [10.132.1.88/23] from ovn-kubernetes 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-b2a05c-684c95bf47-9vx7m Stopping container payload-pre-processing 22m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 22m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 Startup probe failed: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 Startup probe error: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused... 22m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-ppsvj 22m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g Startup probe error: Get "http://10.132.1.87:15021/healthz/ready": dial tcp 10.132.1.87:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g Startup probe failed: Get "http://10.132.1.87:15021/healthz/ready": dial tcp 10.132.1.87:15021: connect: connection refused 22m Normal ScalingReplicaSet deployment/e2e-ipp-b-b2a05c-openshift-default Scaled up replica set e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 from 1 to 4 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-b2a05c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 22m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 Startup probe failed: Get "http://10.132.1.88:15021/healthz/ready": dial tcp 10.132.1.88:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 Startup probe error: Get "http://10.132.1.88:15021/healthz/ready": dial tcp 10.132.1.88:15021: connect: connection refused... 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss 22m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 22m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 22m Normal AddedInterface pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj Add eth0 [10.132.1.90/23] from ovn-kubernetes 22m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Add eth0 [10.133.0.167/23] from ovn-kubernetes 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-e4f8a6-openshift-default Scaled up replica set e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc from 1 to 2 22m Normal Created pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss Container created 22m Normal Started pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj Container started 22m Normal Started pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss Container started 22m Normal Pulled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc Created pod: e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp 22m Normal Created pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj Container created 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-e4f8a6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 22m Normal Pulled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Container created 22m Normal AddedInterface pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss Add eth0 [10.134.0.145/23] from ovn-kubernetes 22m Normal Started pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 Container started 22m Normal Created pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 Container created 22m Normal Pulled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 Add eth0 [10.133.0.166/23] from ovn-kubernetes 22m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 22m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Container started 22m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-add1bb-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-add1bb-openshift-default Scaled up replica set e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 from 2 to 4 22m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 Created pod: e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 Created pod: e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm 22m Normal AddedInterface pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp Add eth0 [10.132.1.91/23] from ovn-kubernetes 22m Normal Pulled pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp Container created 22m Normal Started pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp Container started 22m Normal AddedInterface pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm Add eth0 [10.134.0.146/23] from ovn-kubernetes 22m Normal Pulled pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm Container created 22m Normal Started pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm Container started 22m Warning ProbeError pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Readiness probe error: Get "http://10.132.1.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning Unhealthy pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Readiness probe failed: Get "http://10.132.1.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning Unhealthy pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused 22m Warning ProbeError pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 Startup probe error: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused... 22m Warning ProbeError pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 Startup probe failed: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-18a189-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Created pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-2be695-openshift-default Scaled up replica set e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 from 9 to 10 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9 Created pod: e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-18a189-openshift-default Scaled up replica set e2e-shared-a-w2-18a189-openshift-default-76698f9dc9 from 2 to 3 22m Warning ProbeError pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj Startup probe error: Get "http://10.132.1.90:15021/healthz/ready": dial tcp 10.132.1.90:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj Startup probe failed: Get "http://10.132.1.90:15021/healthz/ready": dial tcp 10.132.1.90:15021: connect: connection refused 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-d81f0a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f Created pod: e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 22m Normal Pulled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Container created 22m Normal Started pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Container started 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-d81f0a-openshift-default Scaled up replica set e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f from 1 to 2 22m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Stopping container istio-proxy 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-dca38d-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 22m Normal SuccessfulDelete replicaset/e2e-worker-w5-dca38d-openshift-default-56494c5f9c Deleted pod: e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl 22m Normal ScalingReplicaSet deployment/e2e-worker-w5-dca38d-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-dca38d-openshift-default-56494c5f9c from 3 to 2 22m Warning Unhealthy pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp Startup probe error: Get "http://10.132.1.91:15021/healthz/ready": dial tcp 10.132.1.91:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp Startup probe failed: Get "http://10.132.1.91:15021/healthz/ready": dial tcp 10.132.1.91:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-262230-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc Created pod: e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-262230-openshift-default Scaled up replica set e2e-shared-a-w1-262230-openshift-default-9d45d8bbc from 3 to 4 22m Warning Unhealthy pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning ProbeError pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-mdgsl Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc Created pod: e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-a6b650-openshift-default Scaled up replica set e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc from 0 to 1 22m Normal AddedLabel gateway/e2e-ipp-a-a6b650 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-a6b650 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-b2a05c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-b2a05c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal AddedInterface pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v Add eth0 [10.134.0.147/23] from ovn-kubernetes 22m Warning ProbeError pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Startup probe error: Get "http://10.132.1.67:15021/healthz/ready": dial tcp 10.132.1.67:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Startup probe failed: Get "http://10.132.1.67:15021/healthz/ready": dial tcp 10.132.1.67:15021: connect: connection refused 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a6b650 Scaled up replica set payload-processing-e2e-ipp-a-a6b650-8dd9774d8 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a6b650 Scaled up replica set payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675 Created pod: payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-r87zd 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a6b650-8dd9774d8 Error creating: pods "payload-processing-e2e-ipp-a-a6b650-8dd9774d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a6b650: serviceaccount "payload-processing-e2e-ipp-a-a6b650" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a6b650-8dd9774d8 Created pod: payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675 Error creating: pods "payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a6b650: serviceaccount "payload-processing-e2e-ipp-a-a6b650" not found 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-r87zd Add eth0 [10.133.0.168/23] from ovn-kubernetes 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-r87zd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-r87zd Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-r87zd Container started 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-b2a05c Scaled up replica set payload-processing-e2e-ipp-b-b2a05c-5d998c694b from 0 to 1 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475 Error creating: pods "payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-b2a05c: serviceaccount "payload-processing-e2e-ipp-b-b2a05c" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-b2a05c Scaled up replica set payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475 from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-b2a05c-5d998c694b Error creating: pods "payload-processing-e2e-ipp-b-b2a05c-5d998c694b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-b2a05c: serviceaccount "payload-processing-e2e-ipp-b-b2a05c" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475 Created pod: payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-b2a05c-5d998c694b Created pod: payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj 22m Warning Unhealthy pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Readiness probe failed: Get "http://10.132.1.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Normal AddedLabel gateway/e2e-ipp-a-368bbb Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-368bbb 22m Warning ProbeError pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Readiness probe error: Get "http://10.132.1.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-368bbb-openshift-default-7bb67896f Created pod: e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-368bbb-openshift-default Scaled up replica set e2e-ipp-a-368bbb-openshift-default-7bb67896f from 0 to 1 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-b2a05c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-b2a05c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal Killing pod/payload-processing-e2e-ipp-b-7f0263-668cfc5fcd-k7kp5 Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-7f0263-b5fd7454c-4s82z Stopping container payload-pre-processing 21m Normal Created pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 Container created 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 Add eth0 [10.132.1.94/23] from ovn-kubernetes 21m Normal Started pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 Container started 21m Normal ScalingReplicaSet deployment/e2e-worker-w1-c104b8-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-c104b8-openshift-default-6fdc484f75 from 3 to 2 21m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Stopping container istio-proxy 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c104b8-openshift-default New size: 2; reason: All metrics below target 21m Normal SuccessfulDelete replicaset/e2e-worker-w1-c104b8-openshift-default-6fdc484f75 Deleted pod: e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 21m Normal Pulled pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-64wz6 Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Started pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Container started 21m Normal Created pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Container created 21m Normal Pulled pod/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-cttw4 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq 21m Normal ScalingReplicaSet deployment/e2e-ipp-b-b2a05c-openshift-default Scaled up replica set e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 from 4 to 8 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-b2a05c-openshift-default New size: 8; reason: 21m Normal Started pod/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj Container started 21m Normal Created pod/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj Container created 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-rlp5s 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wzwfm 21m Normal AddedInterface pod/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj Add eth0 [10.134.0.148/23] from ovn-kubernetes 21m Warning Unhealthy pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Startup probe failed: Get "http://10.132.1.77:15021/healthz/ready": dial tcp 10.132.1.77:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Startup probe error: Get "http://10.132.1.77:15021/healthz/ready": dial tcp 10.132.1.77:15021: connect: connection refused... 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-e92648-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e92648-openshift-default Scaled up replica set e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 from 8 to 10 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 (combined from similar events): Created pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-b2a05c-openshift-default New size: 9; reason: All metrics below target 21m Normal ScalingReplicaSet deployment/e2e-ipp-b-b2a05c-openshift-default Scaled up replica set e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 from 8 to 9 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575 Created pod: e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-6c62j 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff Error creating: pods "payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7f0263: serviceaccount "payload-processing-e2e-ipp-b-7f0263" not found 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-7f0263-5748487fcb Error creating: pods "payload-processing-e2e-ipp-b-7f0263-5748487fcb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7f0263: serviceaccount "payload-processing-e2e-ipp-b-7f0263" not found 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff Created pod: payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-7f0263 Scaled up replica set payload-processing-e2e-ipp-b-7f0263-5748487fcb from 0 to 1 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-7f0263 Scaled up replica set payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-7f0263-5748487fcb Created pod: payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-7f0263-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-7f0263-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal Pulled pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v Container created 21m Normal Started pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v Container started 21m Normal Started pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Container started 21m Normal Created pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Container created 21m Normal Pulled pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Killing pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq Stopping container istio-proxy 21m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-eacea3-openshift-default Scaled down replica set e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f from 3 to 2 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-eacea3-openshift-default New size: 2; reason: All metrics below target 21m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f Deleted pod: e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-cfnpq 21m Warning ProbeError pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v Startup probe error: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v Startup probe failed: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Startup probe error: Get "http://10.132.1.81:15021/healthz/ready": dial tcp 10.132.1.81:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Startup probe failed: Get "http://10.132.1.81:15021/healthz/ready": dial tcp 10.132.1.81:15021: connect: connection refused 21m Normal AddedInterface pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d Add eth0 [10.133.0.169/23] from ovn-kubernetes 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-d3ed60-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-d3ed60-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Killing pod/payload-processing-e2e-ipp-a-d32079-65bd949f84-bxcqs Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-d32079-86d7564b5f-wvddw Stopping container payload-pre-processing 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z Add eth0 [10.134.0.149/23] from ovn-kubernetes 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb Add eth0 [10.134.0.150/23] from ovn-kubernetes 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z Container started 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw Add eth0 [10.134.0.151/23] from ovn-kubernetes 20m Normal Pulled pod/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw Container created 20m Normal Started pod/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw Container started 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z Container created 20m Normal Pulling pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-368bbb Scaled up replica set payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95 from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b Created pod: payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-49xsh 20m Normal Pulled pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.108s (2.108s including waiting). Image size: 125317101 bytes. 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b Error creating: pods "payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-368bbb: serviceaccount "payload-processing-e2e-ipp-a-368bbb" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95 Created pod: payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-q47wt 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95 Error creating: pods "payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-368bbb: serviceaccount "payload-processing-e2e-ipp-a-368bbb" not found 20m Normal Created pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb Container created 20m Normal Started pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb Container started 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-368bbb Scaled up replica set payload-processing-e2e-ipp-a-368bbb-755f6c9c4b from 0 to 1 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-q47wt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-q47wt Add eth0 [10.132.1.98/23] from ovn-kubernetes 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-49xsh Add eth0 [10.132.1.97/23] from ovn-kubernetes 20m Normal Pulled pod/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-49xsh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-49xsh Container started 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-q47wt Container created 20m Normal Created pod/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-49xsh Container created 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-q47wt Container started 20m Normal Killing pod/e2e-ipp-a-d32079-openshift-default-7b55d86997-4cjvp Stopping container istio-proxy 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-d3ed60-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-d3ed60-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Normal Pulled pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq Add eth0 [10.132.1.101/23] from ovn-kubernetes 20m Normal Created pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq Container created 20m Normal Started pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq Container started 20m Warning Unhealthy pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq Startup probe failed: Get "http://10.132.1.101:15021/healthz/ready": dial tcp 10.132.1.101:15021: connect: connection refused 20m Warning ProbeError pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq Startup probe error: Get "http://10.132.1.101:15021/healthz/ready": dial tcp 10.132.1.101:15021: connect: connection refused... 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a6b650-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a6b650-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal AddedLabel gateway/e2e-ipp-a-29bb56 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-29bb56 20m Normal ScalingReplicaSet deployment/e2e-ipp-a-29bb56-openshift-default Scaled up replica set e2e-ipp-a-29bb56-openshift-default-54bf4bb879 from 0 to 1 20m Normal SuccessfulCreate replicaset/e2e-ipp-a-29bb56-openshift-default-54bf4bb879 Created pod: e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-b2a05c-5d998c694b-6lglj Stopping container payload-processing 19m Normal AddedInterface pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Add eth0 [10.134.0.153/23] from ovn-kubernetes 19m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-b-b2a05c-9d7464475-9p5s4 Readiness probe failed: dial tcp 10.132.1.94:9004: i/o timeout 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6 Error creating: pods "payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-29bb56: serviceaccount "payload-processing-e2e-ipp-a-29bb56" not found 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c Error creating: pods "payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-29bb56: serviceaccount "payload-processing-e2e-ipp-a-29bb56" not found 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-29bb56 Scaled up replica set payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c from 0 to 1 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-29bb56 Scaled up replica set payload-processing-e2e-ipp-a-29bb56-57fb68ccf6 from 0 to 1 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6 Created pod: payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c Created pod: payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-rht6s 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-rht6s Add eth0 [10.132.1.106/23] from ovn-kubernetes 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-rht6s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal Created pod/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-rht6s Container created 19m Normal Started pod/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-rht6s Container started 19m Normal Killing pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-wb4ss Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-kf666 Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-ph9vj Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-hbdwq Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-b-b2a05c-openshift-default-dcbdf5575-dhwd6 Stopping container istio-proxy 19m Normal Pulled pod/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq Add eth0 [10.133.0.170/23] from ovn-kubernetes 19m Normal Created pod/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq Container created 19m Normal Started pod/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq Container started 19m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg Container created 19m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v Container started 19m Normal Created pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v Container created 19m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v Add eth0 [10.132.1.109/23] from ovn-kubernetes 19m Normal Started pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg Container started 19m Normal Pulled pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg Add eth0 [10.132.1.108/23] from ovn-kubernetes 19m Normal Killing pod/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l Stopping container payload-pre-processing 19m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v Startup probe failed: Get "http://10.132.1.109:15021/healthz/ready": dial tcp 10.132.1.109:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v Startup probe error: Get "http://10.132.1.109:15021/healthz/ready": dial tcp 10.132.1.109:15021: connect: connection refused... 19m Warning ProbeError pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg Startup probe error: Get "http://10.132.1.108:15021/healthz/ready": dial tcp 10.132.1.108:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg Startup probe failed: Get "http://10.132.1.108:15021/healthz/ready": dial tcp 10.132.1.108:15021: connect: connection refused 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a6b650-7f4b88d675-r87zd Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-a-a6b650-8dd9774d8-shwkb Stopping container payload-processing 19m Normal ScalingReplicaSet deployment/e2e-ipp-b-eaf753-openshift-default Scaled up replica set e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc from 0 to 1 19m Normal AddedLabel gateway/e2e-ipp-b-eaf753 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-eaf753 19m Warning ProbeError pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fqncw Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal SuccessfulCreate replicaset/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc Created pod: e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr 19m Normal Killing pod/e2e-shared-a-w0-eacea3-openshift-default-68bb98d95f-fpj88 Stopping container istio-proxy 19m Normal Killing pod/payload-processing-e2e-ipp-a-368bbb-755f6c9c4b-49xsh Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-368bbb-7b4f867b95-q47wt Stopping container payload-pre-processing 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-eacea3-6579cb8b7-cpm9l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-eacea3-5dddf77dd-2lddq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal AddedInterface pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc Add eth0 [10.132.1.114/23] from ovn-kubernetes 19m Normal AddedInterface pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Add eth0 [10.134.0.157/23] from ovn-kubernetes 19m Normal Created pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc Container created 19m Normal Started pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc Container started 19m Normal Pulled pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp Add eth0 [10.132.1.115/23] from ovn-kubernetes 19m Normal Pulled pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Started pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Container started 19m Normal Pulled pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp Container created 19m Normal Started pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp Container started 19m Normal AddedInterface pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Add eth0 [10.133.0.171/23] from ovn-kubernetes 19m Normal Pulled pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Container created 19m Normal Started pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d Container started 19m Normal Created pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d Container created 19m Normal Killing pod/payload-processing-e2e-ipp-a-4be164-646f4df6bb-8bxsj Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph Stopping container payload-pre-processing 19m Normal AddedInterface pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 Add eth0 [10.134.0.159/23] from ovn-kubernetes 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-73831d-58cbc7867f-tcrmb Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-a-73831d-75bf4f98d-2lkh5 Stopping container payload-processing 19m Normal AddedInterface pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Add eth0 [10.134.0.158/23] from ovn-kubernetes 19m Normal Pulled pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Pulled pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Container created 19m Normal Started pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 Container started 19m Normal Created pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 Container created 19m Normal Started pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Container started 19m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct Readiness probe failed: timeout: failed to connect service "10.132.1.83:50052" within 1s: context deadline exceeded 19m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-a-4be164-7675ddb75-cl2ph Readiness probe failed: dial tcp 10.132.1.85:9004: i/o timeout 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8 Error creating: pods "payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-eaf753: serviceaccount "payload-processing-e2e-ipp-b-eaf753" not found 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-eaf753 Scaled up replica set payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9 from 0 to 1 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9 Created pod: payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-sc42t 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-eaf753 Scaled up replica set payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8 from 0 to 1 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8 Created pod: payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-rrlcc 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9 Error creating: pods "payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-eaf753: serviceaccount "payload-processing-e2e-ipp-b-eaf753" not found 19m Normal Killing pod/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db Stopping container payload-processing 19m Normal AddedInterface pod/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-rrlcc Add eth0 [10.133.0.172/23] from ovn-kubernetes 19m Warning ProbeError pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc Startup probe error: Get "http://10.132.1.114:15021/healthz/ready": dial tcp 10.132.1.114:15021: connect: connection refused... 19m Normal Started pod/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-rrlcc Container started 19m Warning Unhealthy pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc Startup probe failed: Get "http://10.132.1.114:15021/healthz/ready": dial tcp 10.132.1.114:15021: connect: connection refused 19m Normal Pulled pod/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-rrlcc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal Created pod/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-rrlcc Container created 19m Warning ProbeError pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d Startup probe error: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d Startup probe failed: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Startup probe error: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Startup probe failed: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-69dc90-76cd767bf5-7fj4h Stopping container payload-pre-processing 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-368bbb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-368bbb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning Unhealthy pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 Startup probe failed: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Startup probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp Startup probe failed: Get "http://10.132.1.115:15021/healthz/ready": dial tcp 10.132.1.115:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp Startup probe error: Get "http://10.132.1.115:15021/healthz/ready": dial tcp 10.132.1.115:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Startup probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 Startup probe error: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused... 19m Normal Created pod/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-sc42t Container created 19m Normal Started pod/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-sc42t Container started 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-sc42t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-sc42t Add eth0 [10.132.1.117/23] from ovn-kubernetes 19m Normal ScalingReplicaSet deployment/e2e-ipp-a-4be164-openshift-default Scaled up replica set e2e-ipp-a-4be164-openshift-default-7446b5bdf9 from 1 to 2 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-4be164-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 19m Normal SuccessfulCreate replicaset/e2e-ipp-a-4be164-openshift-default-7446b5bdf9 Created pod: e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz 19m Normal Pulled pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Add eth0 [10.132.1.118/23] from ovn-kubernetes 19m Normal Created pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Container created 19m Normal Started pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Container started 18m Normal Pulled pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Created pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Container created 18m Normal Started pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Container started 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-d3ed60-55b9dc7ff-zzxct Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-d3ed60-577c9b6d4d-dbpp4 Stopping container payload-processing 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-eaf753-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-eaf753-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning ProbeError pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Startup probe error: Get "http://10.132.1.118:15021/healthz/ready": dial tcp 10.132.1.118:15021: connect: connection refused... 18m Warning Unhealthy pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Startup probe failed: Get "http://10.132.1.118:15021/healthz/ready": dial tcp 10.132.1.118:15021: connect: connection refused 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-368bbb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-73831d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-368bbb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-73831d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-69dc90-6b769d56d8-v49db Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Startup probe failed: Get "http://10.132.1.80:15021/healthz/ready": dial tcp 10.132.1.80:15021: connect: connection refused 18m Warning ProbeError pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Startup probe error: Get "http://10.132.1.80:15021/healthz/ready": dial tcp 10.132.1.80:15021: connect: connection refused... 18m Warning ProbeError pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Startup probe error: Get "http://10.132.1.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Startup probe failed: Get "http://10.132.1.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Normal Killing pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz Stopping container istio-proxy 18m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f Deleted pod: e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qpxdz 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-29145c-openshift-default New size: 2; reason: All metrics below target 18m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-29145c-openshift-default Scaled down replica set e2e-shared-b-w5-29145c-openshift-default-56b4496f7f from 3 to 2 18m Normal Killing pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-fp5hj Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w0-69dc90-openshift-default-df55b565f-cfbxw Stopping container istio-proxy 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-368bbb Scaled up replica set payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559 from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-368bbb Scaled up replica set payload-processing-e2e-ipp-a-368bbb-55769f5bb4 from 0 to 1 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-368bbb-55769f5bb4 Error creating: pods "payload-processing-e2e-ipp-a-368bbb-55769f5bb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-368bbb: serviceaccount "payload-processing-e2e-ipp-a-368bbb" not found 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-368bbb-55769f5bb4 Created pod: payload-processing-e2e-ipp-a-368bbb-55769f5bb4-8bhjd 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559 Created pod: payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-s86ld 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559 Error creating: pods "payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-368bbb: serviceaccount "payload-processing-e2e-ipp-a-368bbb" not found 18m Normal Created pod/payload-processing-e2e-ipp-a-368bbb-55769f5bb4-8bhjd Container created 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-s86ld Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Pulled pod/payload-processing-e2e-ipp-a-368bbb-55769f5bb4-8bhjd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-processing-e2e-ipp-a-368bbb-55769f5bb4-8bhjd Container started 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-s86ld Add eth0 [10.134.0.161/23] from ovn-kubernetes 18m Normal AddedInterface pod/payload-processing-e2e-ipp-a-368bbb-55769f5bb4-8bhjd Add eth0 [10.134.0.162/23] from ovn-kubernetes 18m Normal Started pod/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-s86ld Container started 18m Normal Created pod/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-s86ld Container created 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-73831d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-73831d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Normal Killing pod/e2e-ipp-a-a6b650-openshift-default-6bc5bcc7dc-z2d6v Stopping container istio-proxy 18m Normal Killing pod/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp Stopping container payload-processing 18m Normal Killing pod/e2e-ipp-a-73831d-openshift-default-d94dc898c-jgf8w Stopping container istio-proxy 18m Normal Killing pod/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc Stopping container payload-pre-processing 18m Normal Killing pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-hrlqb Stopping container istio-proxy 18m Normal Killing pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Stopping container istio-proxy 18m Normal Killing pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Stopping container istio-proxy 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-e2d6d3-777444d8ff-mrztt Stopping container payload-pre-processing 18m Warning Unhealthy pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-ipp-a-4be164-openshift-default-7446b5bdf9-68vtz Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/payload-processing-e2e-shared-a-w4-e2d6d3-5bdbcf5d6d-9z2kk Stopping container payload-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-29bb56-57fb68ccf6-vpkgq Stopping container payload-processing 18m Normal Killing pod/payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk Stopping container payload-pre-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-29bb56-65846f6d8c-rht6s Stopping container payload-pre-processing 18m Warning ProbeError pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-ipp-a-d3ed60-openshift-default-7888cd9c47-gj2mj Readiness probe failed: HTTP probe failed with statuscode: 503 18m Normal SuccessfulCreate replicaset/e2e-ipp-b-679bd2-openshift-default-fb6c76fc Created pod: e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx 18m Normal ScalingReplicaSet deployment/e2e-ipp-b-679bd2-openshift-default Scaled up replica set e2e-ipp-b-679bd2-openshift-default-fb6c76fc from 0 to 1 18m Normal AddedLabel gateway/e2e-ipp-b-679bd2 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-679bd2 18m Normal AddedInterface pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Add eth0 [10.132.1.127/23] from ovn-kubernetes 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf Stopping container payload-processing 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-679bd2 Scaled up replica set payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-679bd2 Scaled up replica set payload-processing-e2e-ipp-b-679bd2-55ff675f5d from 0 to 1 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-ba3d7b-7666cb7f95-wgmzc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f Created pod: payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-fcnrd 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-679bd2-55ff675f5d Created pod: payload-processing-e2e-ipp-b-679bd2-55ff675f5d-zjh5p 18m Normal Killing pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-drtn5 Stopping container istio-proxy 18m Warning Unhealthy pod/payload-processing-e2e-worker-w0-ba3d7b-69dccb6f88-hjxhp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-679bd2-55ff675f5d Error creating: pods "payload-processing-e2e-ipp-b-679bd2-55ff675f5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-679bd2: serviceaccount "payload-processing-e2e-ipp-b-679bd2" not found 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f Error creating: pods "payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-679bd2: serviceaccount "payload-processing-e2e-ipp-b-679bd2" not found 18m Normal Killing pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-tbk6s Stopping container istio-proxy 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-fcnrd Add eth0 [10.132.1.128/23] from ovn-kubernetes 18m Normal Killing pod/e2e-shared-a-w2-18a189-openshift-default-76698f9dc9-j52fc Stopping container istio-proxy 18m Normal Started pod/payload-processing-e2e-ipp-b-679bd2-55ff675f5d-zjh5p Container started 18m Normal Created pod/payload-processing-e2e-ipp-b-679bd2-55ff675f5d-zjh5p Container created 18m Normal Pulled pod/payload-processing-e2e-ipp-b-679bd2-55ff675f5d-zjh5p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-fcnrd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-fcnrd Container created 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-fcnrd Container started 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-679bd2-55ff675f5d-zjh5p Add eth0 [10.132.1.129/23] from ovn-kubernetes 17m Warning ProbeError pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Readiness probe error: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Readiness probe failed: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-18a189-57d89979-ddjvk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-18a189-d4495f868-nsp68 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 Stopping container payload-pre-processing 17m Normal Created pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Container created 17m Warning Unhealthy pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Readiness probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 17m Normal Created pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Container created 17m Normal Pulled pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Container started 17m Normal Started pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Container started 17m Warning ProbeError pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Readiness probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 17m Normal Pulled pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Warning Unhealthy pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 17m Warning ProbeError pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 17m Warning ProbeError pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-262230-9f68f6684-zfxzf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-262230-6c79cfb8b7-z2wnm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-29bb56 Scaled up replica set payload-processing-e2e-ipp-a-29bb56-b9f5594c8 from 0 to 1 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-29bb56-b9f5594c8 Error creating: pods "payload-processing-e2e-ipp-a-29bb56-b9f5594c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-29bb56: serviceaccount "payload-processing-e2e-ipp-a-29bb56" not found 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-29bb56 Scaled up replica set payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5 from 0 to 1 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-29bb56-b9f5594c8 Created pod: payload-processing-e2e-ipp-a-29bb56-b9f5594c8-kms8b 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5 Error creating: pods "payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-29bb56: serviceaccount "payload-processing-e2e-ipp-a-29bb56" not found 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5 Created pod: payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-fbkj2 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-fbkj2 Add eth0 [10.132.1.131/23] from ovn-kubernetes 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp Stopping container istio-proxy 17m Normal Created pod/payload-processing-e2e-ipp-a-29bb56-b9f5594c8-kms8b Container created 17m Normal AddedInterface pod/payload-processing-e2e-ipp-a-29bb56-b9f5594c8-kms8b Add eth0 [10.132.1.132/23] from ovn-kubernetes 17m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-2be695-openshift-default Scaled down replica set e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 from 10 to 8 17m Normal Started pod/payload-processing-e2e-ipp-a-29bb56-b9f5594c8-kms8b Container started 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Stopping container istio-proxy 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-2be695-openshift-default New size: 8; reason: All metrics below target 17m Normal Pulled pod/payload-processing-e2e-ipp-a-29bb56-b9f5594c8-kms8b Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-fbkj2 Container started 17m Normal Created pod/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-fbkj2 Container created 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-fbkj2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Deleted pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-xftjp 17m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79 Deleted pod: e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh 17m Normal Killing pod/payload-processing-e2e-ipp-b-7f0263-5748487fcb-ttphw Stopping container payload-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-eaf753-75dd8cdcd8-rrlcc Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-7f0263-56bc5d9dff-wnw6z Stopping container payload-pre-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-eaf753-64d664cfc9-sc42t Stopping container payload-pre-processing 17m Normal SuccessfulDelete replicaset/e2e-worker-w2-fd3281-openshift-default-75dc5c9669 Deleted pod: e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd 17m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Stopping container istio-proxy 17m Normal ScalingReplicaSet deployment/e2e-worker-w2-fd3281-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-fd3281-openshift-default-75dc5c9669 from 3 to 2 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-fd3281-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-67bf986f99 from 4 to 3 17m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-262230-openshift-default Scaled down replica set e2e-shared-a-w1-262230-openshift-default-9d45d8bbc from 4 to 3 17m Normal Killing pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Stopping container istio-proxy 17m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-67bf986f99 Deleted pod: maas-default-gateway-openshift-default-67bf986f99-fj9f6 17m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 17m Normal SuccessfulDelete replicaset/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc Deleted pod: e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-262230-openshift-default New size: 3; reason: All metrics below target 17m Normal Killing pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-vrk6k Stopping container istio-proxy 17m Normal Killing pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal Killing pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-76z72 Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-252z5 Stopping container istio-proxy 17m Warning ProbeError pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Stopping container istio-proxy 17m Warning ProbeError pod/e2e-worker-w0-ba3d7b-openshift-default-5d87b99755-tb6zf Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-zqskh Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-hjw86 Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-588vd Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal Killing pod/e2e-shared-a-w4-e2d6d3-openshift-default-6f8596d44c-ddkdp Stopping container istio-proxy 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-fj9f6 Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 Stopping container istio-proxy 17m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 Deleted pod: e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-7gsf5 17m Warning ProbeError pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Stopping container istio-proxy 17m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-add1bb-openshift-default Scaled down replica set e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67 from 4 to 3 17m Warning ProbeError pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Readiness probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-add1bb-openshift-default New size: 3; reason: All metrics below target 17m Warning Unhealthy pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-2hptp Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning Unhealthy pod/e2e-shared-a-w1-262230-openshift-default-9d45d8bbc-57z4b Readiness probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-2be695-7574689bb7-pf5f6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning ProbeError pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-ipp-b-7f0263-openshift-default-6c6d84745c-8zzqp Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-2be695-55dcd95b68-qdsx2 Readiness probe failed: timeout: failed to connect service "10.132.0.168:50052" within 1s: context deadline exceeded 17m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-999ff9-867b845c5f-cg2gj Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-shared-b-w4-999ff9-757d5774cd-rmjfn Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 Stopping container payload-processing 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-eaf753-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-eaf753-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Normal Killing pod/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn Stopping container payload-pre-processing 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-9srvt Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-sf8cx Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-mknfl Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-424hp Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4ngp5 Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-4t85v Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-blbmj Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning ProbeError pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-shared-b-w2-2be695-openshift-default-7c8b685f79-2cxjj Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-eaf753 Scaled up replica set payload-pre-processing-e2e-ipp-b-eaf753-8649859f from 0 to 1 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-eaf753-8649859f Error creating: pods "payload-pre-processing-e2e-ipp-b-eaf753-8649859f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-eaf753: serviceaccount "payload-processing-e2e-ipp-b-eaf753" not found 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-29bb56-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-29bb56-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54 Error creating: pods "payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-eaf753: serviceaccount "payload-processing-e2e-ipp-b-eaf753" not found 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-eaf753 Scaled up replica set payload-processing-e2e-ipp-b-eaf753-79d9bf4b54 from 0 to 1 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-eaf753-8649859f-nml4p Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-eaf753-8649859f-nml4p Add eth0 [10.132.1.138/23] from ovn-kubernetes 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-7sck9 Add eth0 [10.132.1.139/23] from ovn-kubernetes 17m Normal Pulled pod/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-7sck9 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54 Created pod: payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-7sck9 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-eaf753-8649859f Created pod: payload-pre-processing-e2e-ipp-b-eaf753-8649859f-nml4p 17m Normal Started pod/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-7sck9 Container started 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-eaf753-8649859f-nml4p Container started 17m Normal Created pod/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-7sck9 Container created 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-eaf753-8649859f-nml4p Container created 17m Normal Killing pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-9dp4d Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w5-64f472-openshift-default-5b7c8dc98c-gk5cj Stopping container istio-proxy 17m Normal Killing pod/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl Stopping container payload-processing 17m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-add1bb-f9bccd668-s9cp4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-add1bb-699bfcfff5-dbnlw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-64f472-588bdcb8c6-rhgnq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-64f472-77bc5f89cf-pfpfn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-679bd2-bb8bfbb7f-fcnrd Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-b-679bd2-55ff675f5d-zjh5p Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 Stopping container payload-processing 16m Normal Killing pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-nlq5j Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-rklxm Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w4-999ff9-openshift-default-64f4c6f67c-cqzk7 Stopping container istio-proxy 16m Normal Created pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Container created 16m Normal Started pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Container started 16m Normal Pulled pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Deleted pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-886adb-openshift-default New size: 7; reason: All metrics below target 16m Warning ProbeError pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Readiness probe error: HTTP probe failed with statuscode: 503... 16m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 Stopping container istio-proxy 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Deleted pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz 16m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-9ljxz Stopping container istio-proxy 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc Deleted pod: e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tc5q4 16m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-sxg6t Stopping container istio-proxy 16m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-886adb-openshift-default Scaled down replica set e2e-shared-b-w3-886adb-openshift-default-64589cdbfc from 10 to 7 16m Warning Unhealthy pod/e2e-shared-b-w1-add1bb-openshift-default-8bdb47f67-r9w5c Readiness probe failed: HTTP probe failed with statuscode: 503 16m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-e92648-openshift-default Scaled down replica set e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 from 10 to 9 16m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v Stopping container istio-proxy 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-e92648-openshift-default New size: 9; reason: All metrics below target 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45 Deleted pod: e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-dh45v 16m Warning Unhealthy pod/payload-processing-e2e-worker-w2-fd3281-864d489c66-rkxzl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-679bd2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-679bd2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-fd3281-596b5fc97f-hs5xw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning ProbeError pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Startup probe error: Get "http://10.132.1.127:15021/healthz/ready": dial tcp 10.132.1.127:15021: connect: connection refused... 16m Warning Unhealthy pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Startup probe failed: Get "http://10.132.1.127:15021/healthz/ready": dial tcp 10.132.1.127:15021: connect: connection refused 16m Normal Killing pod/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj Stopping container payload-processing 16m Normal Killing pod/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 Stopping container payload-pre-processing 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-29145c-677554878d-d2nx7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-29145c-848dbcdcb7-6gkpt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-679bd2-658f485b5b Error creating: pods "payload-processing-e2e-ipp-b-679bd2-658f485b5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-679bd2: serviceaccount "payload-processing-e2e-ipp-b-679bd2" not found 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-679bd2 Scaled up replica set payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf from 0 to 1 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-679bd2 Scaled up replica set payload-processing-e2e-ipp-b-679bd2-658f485b5b from 0 to 1 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf Created pod: payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-kr794 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf Error creating: pods "payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-679bd2: serviceaccount "payload-processing-e2e-ipp-b-679bd2" not found 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-kr794 Add eth0 [10.132.1.155/23] from ovn-kubernetes 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-679bd2-658f485b5b Created pod: payload-processing-e2e-ipp-b-679bd2-658f485b5b-fbjjm 16m Normal Started pod/payload-processing-e2e-ipp-b-679bd2-658f485b5b-fbjjm Container started 16m Normal Pulled pod/payload-processing-e2e-ipp-b-679bd2-658f485b5b-fbjjm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal Created pod/payload-processing-e2e-ipp-b-679bd2-658f485b5b-fbjjm Container created 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-kr794 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-kr794 Container created 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-679bd2-658f485b5b-fbjjm Add eth0 [10.132.1.156/23] from ovn-kubernetes 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-kr794 Container started 16m Normal Killing pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-dbmrd Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w5-29145c-openshift-default-56b4496f7f-qx4vs Stopping container istio-proxy 16m Normal Killing pod/payload-pre-processing-e2e-ipp-a-368bbb-6d7bffb559-s86ld Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-a-368bbb-55769f5bb4-8bhjd Stopping container payload-processing 16m Warning Unhealthy pod/payload-processing-e2e-worker-w4-d827b7-6d9bd77954-w49tj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-d827b7-9659db7ff-d9sj4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb Stopping container payload-processing 16m Warning Unhealthy pod/payload-processing-e2e-worker-w1-c104b8-6c778f644d-bhmp4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-c104b8-68d8ff57f5-sllj6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/e2e-ipp-a-368bbb-openshift-default-7bb67896f-mq44d Stopping container istio-proxy 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-679bd2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-679bd2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-6n6tj Stopping container istio-proxy 16m Normal Killing pod/e2e-worker-w2-fd3281-openshift-default-75dc5c9669-kclfk Stopping container istio-proxy 16m Warning Unhealthy pod/payload-processing-e2e-worker-w5-dca38d-745fb76666-xb8fb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-dca38d-6f88b7b555-kknsm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/payload-processing-e2e-ipp-b-eaf753-79d9bf4b54-7sck9 Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-eaf753-8649859f-nml4p Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-a-29bb56-b9f5594c8-kms8b Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-a-29bb56-6ddf6df5f5-fbkj2 Stopping container payload-pre-processing 15m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Stopping container istio-proxy 15m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-426h8 Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal Killing pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-trsx9 Stopping container istio-proxy 15m Warning ProbeError pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-worker-w4-d827b7-openshift-default-6b4c44bc5c-sk2lx Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal Killing pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Stopping container istio-proxy 15m Warning ProbeError pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-worker-w1-c104b8-openshift-default-6fdc484f75-kb9nv Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning Unhealthy pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-ipp-b-eaf753-openshift-default-f6fbcd6bc-tfkvr Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/e2e-ipp-a-29bb56-openshift-default-54bf4bb879-mkw7w Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-2s6sp Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w5-dca38d-openshift-default-56494c5f9c-dkbkf Stopping container istio-proxy 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-679bd2-77cfb56cf-kr794 Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-b-679bd2-658f485b5b-fbjjm Stopping container payload-processing 15m Normal Killing pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-4lgfb Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w3-e4f8a6-openshift-default-565bc7bbfc-p7szp Stopping container istio-proxy 15m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-e4f8a6-5cbc554998-npwhh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-e4f8a6-64b7df794c-26vvm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Stopping container istio-proxy 15m Warning Unhealthy pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-ipp-b-679bd2-openshift-default-fb6c76fc-bncwx Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 Stopping container payload-pre-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj Stopping container payload-processing 14m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-q62nk Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-tg72s Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-4q28s Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-t6pjp Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-r4nlx Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-cqzvq Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w3-886adb-openshift-default-64589cdbfc-x5xnl Stopping container istio-proxy 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-886adb-74d5cf5c6c-cql48 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-886adb-5bdf49f978-nft8m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-94l66 Stopping container istio-proxy 14m Warning Unhealthy pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-a-w6-d81f0a-openshift-default-8499d4d47f-6kmt6 Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-d81f0a-8486954d99-6xtgt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-d81f0a-f86b6dc76-ffzwj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh Stopping container payload-processing 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-wp9zx Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-nbf6g Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zm4kb Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-5m8k5 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-fc86v Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-w2jv8 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-vbw6c Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-r4pg6 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w6-e92648-openshift-default-6c4f9f8d45-zzzwg Stopping container istio-proxy 14m Warning Unhealthy pod/payload-processing-e2e-worker-w3-82b647-7b45bddbb6-xq44s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-82b647-546f5847f-d9c8r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-e92648-57477cbddb-fhpwh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-e92648-7965cdbb99-kfbbt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 Stopping container payload-processing 14m Normal Killing pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Stopping container istio-proxy 14m Warning Unhealthy pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-worker-w3-82b647-openshift-default-6747d4d57c-w26ps Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/payload-processing-e2e-worker-w6-c4847c-65f8bb6c56-hd5f9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-c4847c-65c84d5d4c-cr22r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Stopping container istio-proxy 13m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Stopping container istio-proxy 13m Normal Killing pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Stopping container istio-proxy 13m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Readiness probe error: HTTP probe failed with statuscode: 503... 13m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-84snq Readiness probe failed: HTTP probe failed with statuscode: 503 13m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Readiness probe failed: HTTP probe failed with statuscode: 503 13m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Readiness probe error: HTTP probe failed with statuscode: 503... 13m Warning Unhealthy pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-958pd Readiness probe failed: HTTP probe failed with statuscode: 503 13m Warning ProbeError pod/e2e-worker-w6-c4847c-openshift-default-65bf677b7f-kcht4 Readiness probe error: HTTP probe failed with statuscode: 503... 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-67bf986f99 from 3 to 4 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-92m4m 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Add eth0 [10.132.1.175/23] from ovn-kubernetes 13m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Container created 13m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Container started 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Startup probe failed: Get "http://10.132.1.175:15021/healthz/ready": dial tcp 10.132.1.175:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Startup probe error: Get "http://10.132.1.175:15021/healthz/ready": dial tcp 10.132.1.175:15021: connect: connection refused... 12m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 12m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-ppsvj 12m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Stopping container discovery 12m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Readiness probe error: Get "http://10.133.0.167:8080/ready": dial tcp 10.133.0.167:8080: connect: connection refused... 12m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 12m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-ppsvj Readiness probe failed: Get "http://10.133.0.167:8080/ready": dial tcp 10.133.0.167:8080: connect: connection refused 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-scn6w 12m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-fwnrr 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-dzvm2 12m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-67bf986f99 from 4 to 8 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-b88g4 12m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Container started 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Add eth0 [10.132.1.179/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Add eth0 [10.134.0.167/23] from ovn-kubernetes 12m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Container started 12m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Container created 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Add eth0 [10.133.0.180/23] from ovn-kubernetes 12m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Container started 12m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Container created 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Add eth0 [10.133.0.179/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Container created 12m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Container created 12m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Container started 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Startup probe failed: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Startup probe error: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Startup probe failed: Get "http://10.132.1.179:15021/healthz/ready": dial tcp 10.132.1.179:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Startup probe error: Get "http://10.132.1.179:15021/healthz/ready": dial tcp 10.132.1.179:15021: connect: connection refused... 11m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Startup probe failed: Get "http://10.132.1.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Startup probe error: Get "http://10.133.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Startup probe error: Get "http://10.132.1.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-scn6w Startup probe failed: Get "http://10.133.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-67bf986f99 from 8 to 10 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-pc85x 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-67bf986f99 Created pod: maas-default-gateway-openshift-default-67bf986f99-f5pvt 11m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 11m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Container started 11m Normal Started pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Container started 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Add eth0 [10.134.0.168/23] from ovn-kubernetes 11m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Container created 11m Normal Created pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Container created 11m Normal Pulled pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Add eth0 [10.132.1.181/23] from ovn-kubernetes 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Startup probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Startup probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Startup probe failed: Get "http://10.132.1.181:15021/healthz/ready": dial tcp 10.132.1.181:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Startup probe error: Get "http://10.132.1.181:15021/healthz/ready": dial tcp 10.132.1.181:15021: connect: connection refused... 11m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Startup probe error: Get "http://10.134.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Startup probe failed: Get "http://10.134.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Startup probe failed: Get "http://10.132.1.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Startup probe error: Get "http://10.132.1.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Readiness probe error: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-j726m Readiness probe failed: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-7r8hn Readiness probe failed: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 10m Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-j726m Readiness probe error: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 10m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-xq4mm Stopping container discovery 10m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 10m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-xq4mm 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Readiness probe failed: Get "http://10.133.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 9m Normal AddedLabel gateway/e2e-worker-single-main-37ba51 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-37ba51 9m Normal ScalingReplicaSet deployment/e2e-worker-single-main-37ba51-openshift-default Scaled up replica set e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97 from 0 to 1 9m Normal SuccessfulCreate replicaset/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97 Created pod: e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 8m59s Normal AddedInterface pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Add eth0 [10.133.0.181/23] from ovn-kubernetes 8m55s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8b6c8c Error creating: pods "payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8b6c8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-37ba51: serviceaccount "payload-processing-e2e-worker-single-main-37ba51" not found 8m55s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-37ba51-59fcb8d546 Error creating: pods "payload-processing-e2e-worker-single-main-37ba51-59fcb8d546-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-37ba51: serviceaccount "payload-processing-e2e-worker-single-main-37ba51" not found 8m55s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-37ba51 Scaled up replica set payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8b6c8c from 0 to 1 8m55s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-37ba51 Scaled up replica set payload-processing-e2e-worker-single-main-37ba51-59fcb8d546 from 0 to 1 8m54s Normal Created pod/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m Container created 8m54s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m Add eth0 [10.132.1.192/23] from ovn-kubernetes 8m54s Normal Started pod/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m Container started 8m54s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-37ba51-59fcb8d546 Created pod: payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b 8m54s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m54s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b Add eth0 [10.133.0.182/23] from ovn-kubernetes 8m54s Normal Pulled pod/payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m54s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8b6c8c Created pod: payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m 8m54s Normal Created pod/payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b Container created 8m54s Normal Started pod/payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b Container started 8m42s Warning ProbeError pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Readiness probe error: Get "http://10.133.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m42s Warning Unhealthy pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Readiness probe failed: Get "http://10.133.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m41s Normal Started pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Container started 8m41s Normal Pulled pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m41s Normal Created pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Container created 8m34s Warning ProbeError pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 8m34s Warning Unhealthy pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 8m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-37ba51-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-37ba51-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m54s Normal Killing pod/payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b Stopping container payload-processing 7m54s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m Stopping container payload-pre-processing 7m44s Normal Killing pod/e2e-worker-single-main-37ba51-openshift-default-6d96ccdc97z7k62 Stopping container istio-proxy 7m32s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-37ba51-59fcb8d54zlv2b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m32s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-37ba51-6fdd8zrk2m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m32s Normal ScalingReplicaSet deployment/e2e-worker-single-main-b63f8a-openshift-default Scaled up replica set e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9 from 0 to 1 5m32s Normal AddedInterface pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Add eth0 [10.133.0.183/23] from ovn-kubernetes 5m32s Normal SuccessfulCreate replicaset/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9 Created pod: e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms 5m32s Normal AddedLabel gateway/e2e-worker-single-main-b63f8a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-b63f8a 5m27s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-b63f8a Scaled up replica set payload-pre-processing-e2e-worker-single-main-b63f8a-9874c59c5 from 0 to 1 5m27s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdd Error creating: pods "payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-b63f8a: serviceaccount "payload-processing-e2e-worker-single-main-b63f8a" not found 5m27s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-b63f8a Scaled up replica set payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdd from 0 to 1 5m27s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c59c5 Error creating: pods "payload-pre-processing-e2e-worker-single-main-b63f8a-9874c59c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-b63f8a: serviceaccount "payload-processing-e2e-worker-single-main-b63f8a" not found 5m26s Normal Pulled pod/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m26s Normal Started pod/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb Container started 5m26s Normal Created pod/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb Container created 5m26s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m26s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb Add eth0 [10.132.1.198/23] from ovn-kubernetes 5m26s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c59c5 Created pod: payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb 5m26s Normal Started pod/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd Container started 5m26s Normal Created pod/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd Container created 5m26s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd Add eth0 [10.133.0.184/23] from ovn-kubernetes 5m26s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdd Created pod: payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd 5m14s Warning ProbeError pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Readiness probe error: Get "http://10.133.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m14s Warning Unhealthy pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Readiness probe failed: Get "http://10.133.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m13s Normal Started pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Container started 5m13s Normal Pulled pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m13s Normal Created pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Container created 5m7s Warning Unhealthy pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Startup probe failed: Get "http://10.133.0.183:15021/healthz/ready": dial tcp 10.133.0.183:15021: connect: connection refused 5m7s Warning ProbeError pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Startup probe error: Get "http://10.133.0.183:15021/healthz/ready": dial tcp 10.133.0.183:15021: connect: connection refused... 4m32s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-b63f8a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m32s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-b63f8a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m31s Normal Killing pod/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd Stopping container payload-processing 4m30s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb Stopping container payload-pre-processing 4m21s Warning ProbeError pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Readiness probe error: HTTP probe failed with statuscode: 503... 4m21s Warning Unhealthy pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Readiness probe failed: HTTP probe failed with statuscode: 503 4m21s Normal Killing pod/e2e-worker-single-main-b63f8a-openshift-default-6876df6cb9nhtms Stopping container istio-proxy 4m14s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-b63f8a-9874c5bmrb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m14s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-b63f8a-7b566c5bdrdpxd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 3m39s Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-fwnrr Readiness probe error: Get "http://10.132.1.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m39s Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-dzvm2 Readiness probe error: Get "http://10.133.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m27s Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Readiness probe failed: Get "http://10.134.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m27s Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-f5pvt Readiness probe error: Get "http://10.134.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m26s Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Readiness probe failed: Get "http://10.132.1.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m26s Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-pc85x Readiness probe error: Get "http://10.132.1.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m24s Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Readiness probe failed: Get "http://10.134.0.167:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m24s Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-b88g4 Readiness probe error: Get "http://10.134.0.167:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m29s Warning ProbeError pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Readiness probe error: Get "http://10.132.1.175:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m29s Warning Unhealthy pod/maas-default-gateway-openshift-default-67bf986f99-92m4m Readiness probe failed: Get "http://10.132.1.175:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)