=== events in openshift-ingress (2026-10-30T23:13:27Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 47m Normal Scheduled pod/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws Successfully assigned openshift-ingress/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws to ip-10-0-134-125.us-west-2.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-lhj8w to ip-10-0-134-125.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Successfully assigned openshift-ingress/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-fxm8x to ip-10-0-141-194.us-west-2.compute.internal 65m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-qsnv7 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qsnv7 to ip-10-0-134-125.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 Successfully assigned openshift-ingress/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 to ip-10-0-129-125.us-west-2.compute.internal 66m Normal Scheduled pod/kube-auth-proxy-784dfbd64d-7pfnm Successfully assigned openshift-ingress/kube-auth-proxy-784dfbd64d-7pfnm to ip-10-0-134-125.us-west-2.compute.internal 66m Normal Scheduled pod/kube-auth-proxy-784dfbd64d-xnvxq Successfully assigned openshift-ingress/kube-auth-proxy-784dfbd64d-xnvxq to ip-10-0-141-194.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-4kprw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-4kprw to ip-10-0-134-125.us-west-2.compute.internal 40m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-4xwjx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-4xwjx to ip-10-0-129-125.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-5c66s Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-5c66s to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz Successfully assigned openshift-ingress/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz to ip-10-0-141-194.us-west-2.compute.internal 32m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-9qn88 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-9qn88 to ip-10-0-129-125.us-west-2.compute.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 to ip-10-0-129-125.us-west-2.compute.internal 36m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc to ip-10-0-129-125.us-west-2.compute.internal 67m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-f24xf Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-f24xf to ip-10-0-134-125.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Successfully assigned openshift-ingress/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 to ip-10-0-134-125.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-bdh5v to ip-10-0-134-125.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p to ip-10-0-134-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Successfully assigned openshift-ingress/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h to ip-10-0-129-125.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z to ip-10-0-129-125.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-cg4z5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-cg4z5 to ip-10-0-134-125.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-clhnh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-clhnh to ip-10-0-134-125.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j Successfully assigned openshift-ingress/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j to ip-10-0-129-125.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Successfully assigned openshift-ingress/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 to ip-10-0-134-125.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm Successfully assigned openshift-ingress/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm to ip-10-0-134-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Successfully assigned openshift-ingress/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs to ip-10-0-141-194.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-cv64p Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-cv64p to ip-10-0-141-194.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw Successfully assigned openshift-ingress/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw to ip-10-0-134-125.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Successfully assigned openshift-ingress/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 to ip-10-0-129-125.us-west-2.compute.internal 72m Normal Scheduled pod/router-default-86b69957bf-6kffb Successfully assigned openshift-ingress/router-default-86b69957bf-6kffb to ip-10-0-129-125.us-west-2.compute.internal 72m Warning FailedScheduling pod/router-default-86b69957bf-6kffb no nodes available to schedule pods 79m Warning FailedScheduling pod/router-default-86b69957bf-6kffb no nodes available to schedule pods 41m Normal Scheduled pod/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk Successfully assigned openshift-ingress/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj Successfully assigned openshift-ingress/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 Successfully assigned openshift-ingress/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 to ip-10-0-141-194.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Successfully assigned openshift-ingress/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr to ip-10-0-134-125.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c to ip-10-0-141-194.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 Successfully assigned openshift-ingress/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Successfully assigned openshift-ingress/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv to ip-10-0-134-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt Successfully assigned openshift-ingress/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt to ip-10-0-129-125.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-dgkjv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-dgkjv to ip-10-0-129-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v to ip-10-0-129-125.us-west-2.compute.internal 32m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-fj2bq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-fj2bq to ip-10-0-134-125.us-west-2.compute.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-g7jrj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-g7jrj to ip-10-0-141-194.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx Successfully assigned openshift-ingress/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx to ip-10-0-134-125.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Successfully assigned openshift-ingress/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 to ip-10-0-141-194.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 to ip-10-0-134-125.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr to ip-10-0-129-125.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Successfully assigned openshift-ingress/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q to ip-10-0-129-125.us-west-2.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Successfully assigned openshift-ingress/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 to ip-10-0-134-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d to ip-10-0-129-125.us-west-2.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-hfhbd to ip-10-0-141-194.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-l597h to ip-10-0-129-125.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-d57980-644975995-869mp Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d57980-644975995-869mp to ip-10-0-134-125.us-west-2.compute.internal 36m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 to ip-10-0-141-194.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Successfully assigned openshift-ingress/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j to ip-10-0-129-125.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h to ip-10-0-129-125.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-lxnwd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-lxnwd to ip-10-0-141-194.us-west-2.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-m55sg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-m55sg to ip-10-0-141-194.us-west-2.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w4-b13755-557c67d777-rqfvg Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-b13755-557c67d777-rqfvg to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n42cb to ip-10-0-134-125.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-n92qw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-n92qw to ip-10-0-134-125.us-west-2.compute.internal 67m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-nmwcz to ip-10-0-141-194.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn to ip-10-0-134-125.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf to ip-10-0-134-125.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc to ip-10-0-134-125.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-nr4sp to ip-10-0-141-194.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-nwxll to ip-10-0-129-125.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-rwbtf to ip-10-0-129-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf to ip-10-0-141-194.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-v7krd to ip-10-0-134-125.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m to ip-10-0-129-125.us-west-2.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-wgcfm to ip-10-0-129-125.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w Successfully assigned openshift-ingress/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w to ip-10-0-141-194.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc to ip-10-0-134-125.us-west-2.compute.internal 32m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-zldcw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zldcw to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c to ip-10-0-134-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz to ip-10-0-134-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 to ip-10-0-129-125.us-west-2.compute.internal 40m Normal Scheduled pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zz7mk to ip-10-0-134-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d to ip-10-0-129-125.us-west-2.compute.internal 6m27s Normal Scheduled pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Successfully assigned openshift-ingress/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn to ip-10-0-129-125.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 to ip-10-0-141-194.us-west-2.compute.internal 15m Normal Scheduled pod/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k Successfully assigned openshift-ingress/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k to ip-10-0-134-125.us-west-2.compute.internal 50m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qjq6b to ip-10-0-129-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 to ip-10-0-129-125.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 to ip-10-0-141-194.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 to ip-10-0-129-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn to ip-10-0-129-125.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl to ip-10-0-141-194.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m to ip-10-0-141-194.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Successfully assigned openshift-ingress/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc to ip-10-0-141-194.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Successfully assigned openshift-ingress/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 to ip-10-0-141-194.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 to ip-10-0-141-194.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 to ip-10-0-141-194.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s to ip-10-0-134-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 to ip-10-0-129-125.us-west-2.compute.internal 15m Normal Scheduled pod/payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr to ip-10-0-134-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 Successfully assigned openshift-ingress/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 to ip-10-0-129-125.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b to ip-10-0-134-125.us-west-2.compute.internal 10m Normal Scheduled pod/payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r to ip-10-0-134-125.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-4fv8l Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-4fv8l to ip-10-0-129-125.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-d57980-6948d6646c-2hpbb Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-d57980-6948d6646c-2hpbb to ip-10-0-134-125.us-west-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-h6pqc Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-h6pqc to ip-10-0-129-125.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm to ip-10-0-134-125.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-lz4ld Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-lz4ld to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq to ip-10-0-141-194.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 to ip-10-0-141-194.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-dh9xv Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-dh9xv to ip-10-0-141-194.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-926fc6-7776b58857-8d7xh Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-926fc6-7776b58857-8d7xh to ip-10-0-129-125.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-9z2d2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-9z2d2 to ip-10-0-129-125.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-mt-61346ee8-789d6d5f75-xhp6q Successfully assigned openshift-ingress/payload-processing-e2e-mt-61346ee8-789d6d5f75-xhp6q to ip-10-0-141-194.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Successfully assigned openshift-ingress/e2e-ait-dee83803-openshift-default-54775796bf-8vttp to ip-10-0-134-125.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 to ip-10-0-141-194.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b Successfully assigned openshift-ingress/payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b to ip-10-0-129-125.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-ck47n Successfully assigned openshift-ingress/payload-processing-e2e-mt-29bbe735-859dbf5fc-ck47n to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-hwpsq Successfully assigned openshift-ingress/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-hwpsq to ip-10-0-141-194.us-west-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw to ip-10-0-129-125.us-west-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw to ip-10-0-141-194.us-west-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s to ip-10-0-129-125.us-west-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-d6b55f-9d98cc4f-dcmt4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-d6b55f-9d98cc4f-dcmt4 to ip-10-0-141-194.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-sghhb Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-sghhb to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw to ip-10-0-141-194.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-v6zl6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-v6zl6 to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-derive-e8ed1149-755778c777-nnmqv Successfully assigned openshift-ingress/payload-processing-e2e-derive-e8ed1149-755778c777-nnmqv to ip-10-0-129-125.us-west-2.compute.internal 5m20s Normal Scheduled pod/payload-processing-e2e-authcfg-http-152666-5b4bb5865f-2qjwp Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-152666-5b4bb5865f-2qjwp to ip-10-0-129-125.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-api-b-b5313f-685744b7c7-4lhg6 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-b5313f-685744b7c7-4lhg6 to ip-10-0-134-125.us-west-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-api-b-9fce61-7bfc755f9c-vjgk6 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-9fce61-7bfc755f9c-vjgk6 to ip-10-0-129-125.us-west-2.compute.internal 34m Normal Scheduled pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Successfully assigned openshift-ingress/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw to ip-10-0-134-125.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl Successfully assigned openshift-ingress/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl to ip-10-0-129-125.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-api-a-a3600a-5767d5467c-snzfh Successfully assigned openshift-ingress/payload-processing-e2e-api-a-a3600a-5767d5467c-snzfh to ip-10-0-129-125.us-west-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-api-a-771ff9-65d554b547-ct57x Successfully assigned openshift-ingress/payload-processing-e2e-api-a-771ff9-65d554b547-ct57x to ip-10-0-129-125.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-721c75-6b765575f7-zklj8 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-721c75-6b765575f7-zklj8 to ip-10-0-129-125.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 to ip-10-0-129-125.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f to ip-10-0-134-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 to ip-10-0-129-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc to ip-10-0-134-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws to ip-10-0-134-125.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-ait-dee83803-7f495bcb9f-xqbzg Successfully assigned openshift-ingress/payload-processing-e2e-ait-dee83803-7f495bcb9f-xqbzg to ip-10-0-141-194.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 to ip-10-0-141-194.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn to ip-10-0-129-125.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Successfully assigned openshift-ingress/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 to ip-10-0-129-125.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 to ip-10-0-134-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp to ip-10-0-134-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw to ip-10-0-129-125.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf to ip-10-0-134-125.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-ait-ae20a62d-76cbb85fc-2j2qs Successfully assigned openshift-ingress/payload-processing-e2e-ait-ae20a62d-76cbb85fc-2j2qs to ip-10-0-141-194.us-west-2.compute.internal 6m19s Normal Scheduled pod/payload-processing-e2e-ait-6e41e55b-77fbff758f-7xjhc Successfully assigned openshift-ingress/payload-processing-e2e-ait-6e41e55b-77fbff758f-7xjhc to ip-10-0-129-125.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj to ip-10-0-134-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q to ip-10-0-129-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq to ip-10-0-141-194.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v to ip-10-0-129-125.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh to ip-10-0-134-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr to ip-10-0-134-125.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht to ip-10-0-134-125.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-ait-05e3625f-5c977df99-tpplh Successfully assigned openshift-ingress/payload-processing-e2e-ait-05e3625f-5c977df99-tpplh to ip-10-0-141-194.us-west-2.compute.internal 67m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-86dbb48f59-q4pbk Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-86dbb48f59-q4pbk to ip-10-0-129-125.us-west-2.compute.internal 34m Normal Scheduled pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Successfully assigned openshift-ingress/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv to ip-10-0-129-125.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Successfully assigned openshift-ingress/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 to ip-10-0-129-125.us-west-2.compute.internal 62m Normal Scheduled pod/payload-pre-processing-7b98bb678c-t9lxh Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-t9lxh to ip-10-0-129-125.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-gnzwz Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-gnzwz to ip-10-0-141-194.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs to ip-10-0-134-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz to ip-10-0-134-125.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 to ip-10-0-134-125.us-west-2.compute.internal 62m Normal Scheduled pod/payload-processing-55fbbbf9dc-fdzns Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-fdzns to ip-10-0-129-125.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Successfully assigned openshift-ingress/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 to ip-10-0-129-125.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Successfully assigned openshift-ingress/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq to ip-10-0-141-194.us-west-2.compute.internal 66m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-57d8447fc-th9dm Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-57d8447fc-th9dm to ip-10-0-129-125.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 to ip-10-0-129-125.us-west-2.compute.internal 6m19s Normal Scheduled pod/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-nhrtj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-nhrtj to ip-10-0-134-125.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-68fxm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-68fxm to ip-10-0-141-194.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj to ip-10-0-141-194.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj to ip-10-0-134-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 to ip-10-0-141-194.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 to ip-10-0-129-125.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Successfully assigned openshift-ingress/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct to ip-10-0-129-125.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-ait-dee83803-cdc57876c-xfwmv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-dee83803-cdc57876c-xfwmv to ip-10-0-141-194.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Successfully assigned openshift-ingress/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q to ip-10-0-134-125.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn Successfully assigned openshift-ingress/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn to ip-10-0-129-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 to ip-10-0-134-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp to ip-10-0-141-194.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf to ip-10-0-134-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k to ip-10-0-129-125.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm to ip-10-0-129-125.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-721c75-5f454d877f-hr4x7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-721c75-5f454d877f-hr4x7 to ip-10-0-129-125.us-west-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-qz5hj Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-qz5hj to ip-10-0-129-125.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-a-a3600a-684485b7f9-7pnqr Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-a3600a-684485b7f9-7pnqr to ip-10-0-129-125.us-west-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-api-b-9fce61-86b586b655-x4pz8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-9fce61-86b586b655-x4pz8 to ip-10-0-129-125.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-hfnzt Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-hfnzt to ip-10-0-134-125.us-west-2.compute.internal 5m20s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-b7ps4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-b7ps4 to ip-10-0-129-125.us-west-2.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-ph6zx Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-ph6zx to ip-10-0-129-125.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-9jv96 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-9jv96 to ip-10-0-129-125.us-west-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs to ip-10-0-129-125.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Successfully assigned openshift-ingress/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx to ip-10-0-134-125.us-west-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl to ip-10-0-129-125.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Successfully assigned openshift-ingress/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx to ip-10-0-129-125.us-west-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 to ip-10-0-141-194.us-west-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-6ntlk Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-6ntlk to ip-10-0-141-194.us-west-2.compute.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j to ip-10-0-141-194.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn to ip-10-0-129-125.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-tclzz Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-tclzz to ip-10-0-129-125.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-mt-61346ee8-7457f784f5-mcgws Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-61346ee8-7457f784f5-mcgws to ip-10-0-141-194.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Successfully assigned openshift-ingress/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g to ip-10-0-129-125.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x to ip-10-0-141-194.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-5zmbn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-5zmbn to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-27hbh Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-27hbh to ip-10-0-129-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl to ip-10-0-129-125.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-fwdsz Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-fwdsz to ip-10-0-129-125.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-vcbjl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-vcbjl to ip-10-0-129-125.us-west-2.compute.internal 5m29s Normal Scheduled pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Successfully assigned openshift-ingress/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 to ip-10-0-134-125.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-72btg Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-72btg to ip-10-0-129-125.us-west-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 to ip-10-0-129-125.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Successfully assigned openshift-ingress/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd to ip-10-0-141-194.us-west-2.compute.internal 10m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl to ip-10-0-129-125.us-west-2.compute.internal 15m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml to ip-10-0-129-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk to ip-10-0-129-125.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r to ip-10-0-141-194.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js to ip-10-0-129-125.us-west-2.compute.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb to ip-10-0-141-194.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Successfully assigned openshift-ingress/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn to ip-10-0-129-125.us-west-2.compute.internal 79m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-86b69957bf from 0 to 1 79m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 79m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 79m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 79m Normal SuccessfulCreate replicaset/router-default-86b69957bf Created pod: router-default-86b69957bf-6kffb 72m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 72m Warning FailedMount pod/router-default-86b69957bf-6kffb MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 72m Warning FailedMount pod/router-default-86b69957bf-6kffb MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 72m Warning FailedMount pod/router-default-86b69957bf-6kffb MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 72m Warning FailedMount pod/router-default-86b69957bf-6kffb MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 72m Normal AddedInterface pod/router-default-86b69957bf-6kffb Add eth0 [10.132.0.16/23] from ovn-kubernetes 72m Warning Unhealthy pod/router-default-86b69957bf-6kffb Startup probe failed: HTTP probe failed with statuscode: 500 72m Normal Created pod/router-default-86b69957bf-6kffb Container created 72m Warning ProbeError pod/router-default-86b69957bf-6kffb Startup probe error: HTTP probe failed with statuscode: 500... 72m Normal Pulled pod/router-default-86b69957bf-6kffb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 72m Normal Started pod/router-default-86b69957bf-6kffb Container started 68m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 67m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 67m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-f24xf 67m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-f24xf Add eth0 [10.134.0.20/23] from ovn-kubernetes 67m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-f24xf Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 67m Normal Started pod/istiod-openshift-gateway-7876bfcb74-f24xf Container started 67m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-f24xf Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.114s (4.114s including waiting). Image size: 185476477 bytes. 67m Normal Created pod/istiod-openshift-gateway-7876bfcb74-f24xf Container created 67m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-f24xf Readiness probe failed: HTTP probe failed with statuscode: 503 67m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-f24xf Readiness probe error: HTTP probe failed with statuscode: 503... 67m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-86dbb48f59 from 0 to 1 67m Normal Pulling pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 67m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-nmwcz 67m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-86dbb48f59 Created pod: openshift-ai-inference-openshift-ai-inference-86dbb48f59-q4pbk 67m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75bf867b6c from 0 to 1 67m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 67m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Add eth0 [10.133.0.17/23] from ovn-kubernetes 67m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-86dbb48f59-q4pbk Add eth0 [10.132.0.35/23] from ovn-kubernetes 67m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-86dbb48f59-q4pbk Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 67m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 67m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.55s (4.55s including waiting). Image size: 183555333 bytes. 67m Normal Started pod/openshift-ai-inference-openshift-ai-inference-86dbb48f59-q4pbk Container started 67m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-86dbb48f59-q4pbk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.739s (9.739s including waiting). Image size: 183555333 bytes. 67m Normal Created pod/openshift-ai-inference-openshift-ai-inference-86dbb48f59-q4pbk Container created 67m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-784dfbd64d from 0 to 2 66m Normal SuccessfulCreate replicaset/kube-auth-proxy-784dfbd64d Created pod: kube-auth-proxy-784dfbd64d-xnvxq 66m Normal SuccessfulCreate replicaset/kube-auth-proxy-784dfbd64d Created pod: kube-auth-proxy-784dfbd64d-7pfnm 66m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-57d8447fc Created pod: data-science-gateway-data-science-gateway-class-57d8447fc-th9dm 66m Normal AddedInterface pod/kube-auth-proxy-784dfbd64d-xnvxq Add eth0 [10.133.0.25/23] from ovn-kubernetes 66m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-57d8447fc from 0 to 1 66m Normal AddedInterface pod/kube-auth-proxy-784dfbd64d-7pfnm Add eth0 [10.134.0.22/23] from ovn-kubernetes 66m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-57d8447fc-th9dm Add eth0 [10.132.0.43/23] from ovn-kubernetes 66m Normal Started pod/data-science-gateway-data-science-gateway-class-57d8447fc-th9dm Container started 66m Normal Created pod/data-science-gateway-data-science-gateway-class-57d8447fc-th9dm Container created 66m Normal Pulled pod/data-science-gateway-data-science-gateway-class-57d8447fc-th9dm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Pulling pod/kube-auth-proxy-784dfbd64d-7pfnm Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 66m Normal Pulling pod/kube-auth-proxy-784dfbd64d-xnvxq Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 66m Normal Created pod/kube-auth-proxy-784dfbd64d-xnvxq Container created 66m Normal Pulled pod/kube-auth-proxy-784dfbd64d-xnvxq Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.385s (3.385s including waiting). Image size: 188904677 bytes. 66m Normal Started pod/kube-auth-proxy-784dfbd64d-xnvxq Container started 66m Normal Started pod/kube-auth-proxy-784dfbd64d-7pfnm Container started 66m Normal Created pod/kube-auth-proxy-784dfbd64d-7pfnm Container created 66m Normal Pulled pod/kube-auth-proxy-784dfbd64d-7pfnm Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.951s (2.951s including waiting). Image size: 188904677 bytes. 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 65m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-qsnv7 Add eth0 [10.134.0.29/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-qsnv7 65m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-qsnv7 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 65m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-qsnv7 Container created 65m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-qsnv7 Container started 65m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-qsnv7 Readiness probe error: HTTP probe failed with statuscode: 503... 65m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-qsnv7 Readiness probe failed: HTTP probe failed with statuscode: 503 65m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-f24xf 65m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-f24xf Stopping container discovery 65m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 65m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Container started 65m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Container created 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Startup probe failed: Get "http://10.133.0.17:15021/healthz/ready": dial tcp 10.133.0.17:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Startup probe error: Get "http://10.133.0.17:15021/healthz/ready": dial tcp 10.133.0.17:15021: connect: connection refused... 64m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75bf867b6c from 1 to 2 62m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-wgcfm 62m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Container created 62m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Container started 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Add eth0 [10.132.0.53/23] from ovn-kubernetes 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-lhj8w 62m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 62m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75bf867b6c from 2 to 3 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Add eth0 [10.134.0.37/23] from ovn-kubernetes 62m Normal Pulling pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 62m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Container started 62m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Container created 62m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.965s (3.965s including waiting). Image size: 183555333 bytes. 62m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 62m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 62m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 62m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 62m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 62m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-fdzns 62m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-t9lxh 62m Normal Pulling pod/payload-pre-processing-7b98bb678c-t9lxh Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 62m Normal AddedInterface pod/payload-processing-55fbbbf9dc-fdzns Add eth0 [10.132.0.57/23] from ovn-kubernetes 62m Normal Pulling pod/payload-processing-55fbbbf9dc-fdzns Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 62m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-t9lxh Add eth0 [10.132.0.56/23] from ovn-kubernetes 62m Normal Started pod/payload-processing-55fbbbf9dc-fdzns Container started 62m Normal Pulled pod/payload-pre-processing-7b98bb678c-t9lxh Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.192s (3.192s including waiting). Image size: 125353965 bytes. 62m Normal Created pod/payload-pre-processing-7b98bb678c-t9lxh Container created 62m Normal Started pod/payload-pre-processing-7b98bb678c-t9lxh Container started 62m Normal Created pod/payload-processing-55fbbbf9dc-fdzns Container created 62m Normal Pulled pod/payload-processing-55fbbbf9dc-fdzns Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.086s (3.086s including waiting). Image size: 125353965 bytes. 61m Normal AddedLabel gateway/e2e-worker-w1-af30a9 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-af30a9 61m Normal AddedLabel gateway/e2e-worker-w6-be17e7 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-be17e7 61m Normal AddedLabel gateway/e2e-worker-w5-f80a0e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-f80a0e 61m Normal AddedLabel gateway/e2e-worker-w2-eb4aa1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-eb4aa1 61m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Add eth0 [10.132.0.59/23] from ovn-kubernetes 61m Normal AddedLabel gateway/e2e-worker-w0-d2d4c3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-d2d4c3 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled up replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 0 to 1 61m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled up replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 0 to 1 61m Normal AddedLabel gateway/e2e-worker-w3-23f3d4 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-23f3d4 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled up replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 0 to 1 61m Normal AddedLabel gateway/e2e-worker-w4-b13755 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-b13755 61m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 Add eth0 [10.134.0.38/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Created pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 61m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled up replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 0 to 1 61m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Add eth0 [10.132.0.60/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b 61m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled up replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 0 to 1 61m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Add eth0 [10.132.0.61/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/e2e-worker-w6-be17e7-openshift-default Scaled up replica set e2e-worker-w6-be17e7-openshift-default-7c64556c75 from 0 to 1 61m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Add eth0 [10.134.0.39/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Add eth0 [10.134.0.40/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 Add eth0 [10.132.0.62/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled up replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 0 to 1 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f Error creating: pods "payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-b13755: serviceaccount "payload-processing-e2e-worker-w4-b13755" not found 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-b13755 Scaled up replica set payload-processing-e2e-worker-w4-b13755-557c67d777 from 0 to 1 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-b13755 Scaled up replica set payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f from 0 to 1 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f Created pod: payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-b13755-557c67d777 Error creating: pods "payload-processing-e2e-worker-w4-b13755-557c67d777-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-b13755: serviceaccount "payload-processing-e2e-worker-w4-b13755" not found 61m Normal Pulled pod/payload-processing-e2e-worker-w4-b13755-557c67d777-rqfvg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Created pod/payload-processing-e2e-worker-w4-b13755-557c67d777-rqfvg Container created 61m Normal AddedInterface pod/payload-processing-e2e-worker-w4-b13755-557c67d777-rqfvg Add eth0 [10.132.0.64/23] from ovn-kubernetes 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl Add eth0 [10.132.0.63/23] from ovn-kubernetes 61m Normal Created pod/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl Container created 61m Normal Started pod/payload-processing-e2e-worker-w4-b13755-557c67d777-rqfvg Container started 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-b13755-557c67d777 Created pod: payload-processing-e2e-worker-w4-b13755-557c67d777-rqfvg 61m Normal Started pod/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl Container started 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c Error creating: pods "payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-af30a9: serviceaccount "payload-processing-e2e-worker-w1-af30a9" not found 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7 Error creating: pods "payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-af30a9: serviceaccount "payload-processing-e2e-worker-w1-af30a9" not found 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-af30a9 Scaled up replica set payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7 from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7 Created pod: payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-af30a9 Scaled up replica set payload-processing-e2e-worker-w1-af30a9-b44c8fc5c from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c Created pod: payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn 60m Normal Created pod/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn Container created 60m Normal Started pod/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk Container started 60m Normal Started pod/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn Container started 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Normal Created pod/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk Container created 60m Normal Pulled pod/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk Add eth0 [10.132.0.65/23] from ovn-kubernetes 60m Normal AddedInterface pod/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn Add eth0 [10.132.0.66/23] from ovn-kubernetes 60m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c Created pod: payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c Error creating: pods "payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-23f3d4: serviceaccount "payload-processing-e2e-worker-w3-23f3d4" not found 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-23f3d4 Scaled up replica set payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c from 0 to 1 60m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45 Error creating: pods "payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-23f3d4: serviceaccount "payload-processing-e2e-worker-w3-23f3d4" not found 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-23f3d4 Scaled up replica set payload-processing-e2e-worker-w3-23f3d4-6b7474cb45 from 0 to 1 60m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j Add eth0 [10.133.0.34/23] from ovn-kubernetes 60m Normal Pulling pod/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal AddedInterface pod/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf Add eth0 [10.133.0.35/23] from ovn-kubernetes 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45 Created pod: payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-f80a0e Scaled up replica set payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf from 0 to 1 60m Normal Pulling pod/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675 Created pod: payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k 60m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf Error creating: pods "payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-f80a0e: serviceaccount "payload-processing-e2e-worker-w5-f80a0e" not found 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-f80a0e Scaled up replica set payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675 from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf Created pod: payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675 Error creating: pods "payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-f80a0e: serviceaccount "payload-processing-e2e-worker-w5-f80a0e" not found 60m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.034s (3.034s including waiting). Image size: 125353965 bytes. 60m Normal Pulled pod/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Started pod/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d Container started 60m Normal Started pod/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k Container started 60m Normal Created pod/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d Container created 60m Normal Created pod/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k Container created 60m Normal Started pod/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j Container started 60m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k Add eth0 [10.132.0.67/23] from ovn-kubernetes 60m Normal Created pod/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j Container created 60m Normal AddedInterface pod/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d Add eth0 [10.132.0.68/23] from ovn-kubernetes 60m Normal Started pod/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf Container started 60m Normal Created pod/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf Container created 60m Normal Pulled pod/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.992s (2.992s including waiting). Image size: 125353965 bytes. 60m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-be17e7-546f498b5c Error creating: pods "payload-processing-e2e-worker-w6-be17e7-546f498b5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-be17e7: serviceaccount "payload-processing-e2e-worker-w6-be17e7" not found 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc Error creating: pods "payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-be17e7: serviceaccount "payload-processing-e2e-worker-w6-be17e7" not found 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-be17e7 Scaled up replica set payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc from 0 to 1 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-be17e7 Scaled up replica set payload-processing-e2e-worker-w6-be17e7-546f498b5c from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc Created pod: payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-be17e7-546f498b5c Created pod: payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v 60m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Created pod/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 Container created 60m Normal Started pod/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 Container started 60m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 Add eth0 [10.133.0.36/23] from ovn-kubernetes 60m Normal Created pod/payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v Container created 60m Normal Pulled pod/payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Started pod/payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v Container started 60m Normal AddedInterface pod/payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v Add eth0 [10.132.0.69/23] from ovn-kubernetes 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7 Created pod: payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-eb4aa1 Scaled up replica set payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc from 0 to 1 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc Error creating: pods "payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-eb4aa1: serviceaccount "payload-processing-e2e-worker-w2-eb4aa1" not found 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc Created pod: payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-eb4aa1 Scaled up replica set payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7 from 0 to 1 60m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7 Error creating: pods "payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-eb4aa1: serviceaccount "payload-processing-e2e-worker-w2-eb4aa1" not found 60m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl Add eth0 [10.132.0.70/23] from ovn-kubernetes 60m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Container started 60m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Container created 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Normal AddedInterface pod/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 Add eth0 [10.132.0.71/23] from ovn-kubernetes 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-d2d4c3 Scaled up replica set payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9 from 0 to 1 60m Normal Created pod/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl Container created 60m Normal Started pod/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl Container started 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-d2d4c3 Scaled up replica set payload-processing-e2e-worker-w0-d2d4c3-847ddbf687 from 0 to 1 60m Normal Started pod/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 Container started 60m Normal Created pod/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 Container created 60m Normal Pulled pod/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9 Created pod: payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687 Created pod: payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 60m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687 Error creating: pods "payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-d2d4c3: serviceaccount "payload-processing-e2e-worker-w0-d2d4c3" not found 60m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb Add eth0 [10.133.0.38/23] from ovn-kubernetes 60m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9 Error creating: pods "payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-d2d4c3: serviceaccount "payload-processing-e2e-worker-w0-d2d4c3" not found 60m Normal AddedInterface pod/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 Add eth0 [10.132.0.72/23] from ovn-kubernetes 60m Normal Created pod/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb Container created 60m Normal Started pod/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb Container started 60m Normal Pulled pod/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Created pod/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 Container created 60m Normal Started pod/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 Container started 60m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused 60m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Container created 60m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Container started 60m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Container created 60m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Container started 60m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Readiness probe failed: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Readiness probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Readiness probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Readiness probe error: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Container started 60m Normal Created pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Container created 60m Normal Pulled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Container started 60m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Container created 60m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 Container created 60m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 Container started 60m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 Container created 59m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 Container started 59m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused... 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled up replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 1 to 2 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc 59m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc Add eth0 [10.134.0.47/23] from ovn-kubernetes 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc Container started 59m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc Container created 59m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled up replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 1 to 2 58m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Container created 58m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Container started 58m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Add eth0 [10.134.0.48/23] from ovn-kubernetes 58m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Startup probe failed: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Startup probe error: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused... 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled up replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 2 to 3 57m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Container created 57m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Add eth0 [10.133.0.51/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Container started 57m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled up replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 1 to 2 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 57m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Add eth0 [10.132.0.83/23] from ovn-kubernetes 57m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Container started 57m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Container created 57m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 57m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-nmwcz 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75bf867b6c from 3 to 2 57m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-nmwcz Stopping container istio-proxy 56m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled up replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 3 to 4 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v 56m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Container created 56m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled up replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 2 to 3 56m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Add eth0 [10.132.0.86/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 56m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Container started 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 56m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Add eth0 [10.133.0.52/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Container created 56m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Container started 56m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 56m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled up replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 1 to 2 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f 56m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f Add eth0 [10.134.0.51/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f Container started 56m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f Container created 56m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-g7jrj 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-bdcn8 56m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 Container started 56m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-g7jrj Container created 56m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-g7jrj Container started 56m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 Container created 56m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 Add eth0 [10.132.0.96/23] from ovn-kubernetes 56m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-g7jrj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-g7jrj Add eth0 [10.133.0.53/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled up replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 3 to 4 55m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 55m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Container started 55m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Container created 55m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Add eth0 [10.132.0.99/23] from ovn-kubernetes 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-g7jrj Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-g7jrj Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-clhnh 55m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75bf867b6c from 4 to 6 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-cv64p 55m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-clhnh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-clhnh Container started 55m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-cv64p Container created 55m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-cv64p Container started 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-clhnh Add eth0 [10.134.0.53/23] from ovn-kubernetes 55m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-cv64p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-cv64p Add eth0 [10.133.0.54/23] from ovn-kubernetes 55m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-clhnh Container created 55m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-cv64p Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-clhnh Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-cv64p Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-clhnh Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 55m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Container created 55m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Add eth0 [10.133.0.55/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x 55m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled up replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 2 to 4 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 55m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Container started 55m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Add eth0 [10.134.0.54/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled up replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 4 to 6 55m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Container created 55m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Add eth0 [10.132.0.106/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Container started 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-be17e7-openshift-default Scaled up replica set e2e-worker-w6-be17e7-openshift-default-7c64556c75 from 1 to 2 55m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Container created 55m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Container started 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Created pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 55m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Add eth0 [10.133.0.56/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Container started 55m Normal AddedInterface pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Add eth0 [10.132.0.107/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Container created 55m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Container created 55m Normal Started pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Container started 55m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled up replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 4 to 6 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 54m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Container created 54m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Container started 54m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Container started 54m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Add eth0 [10.133.0.59/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled up replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 1 to 2 54m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Add eth0 [10.134.0.58/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Container created 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj 54m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Container created 54m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Add eth0 [10.134.0.59/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Container started 54m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled up replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 1 to 2 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm 54m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Add eth0 [10.132.0.109/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Container created 54m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Container started 54m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled up replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 2 to 3 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 Container created 53m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 Container started 53m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 Add eth0 [10.133.0.61/23] from ovn-kubernetes 53m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled up replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 2 to 3 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 52m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Container started 52m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Add eth0 [10.133.0.62/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Container created 52m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled up replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 6 to 10 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 (combined from similar events): Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Created pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw 51m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Created pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs 51m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz Add eth0 [10.134.0.62/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Container started 51m Normal Created pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Container created 51m Normal Pulled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Add eth0 [10.133.0.65/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 Container started 51m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 Container created 51m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 Add eth0 [10.132.0.118/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz Container created 51m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz Container started 51m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Container started 51m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Container created 51m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Add eth0 [10.133.0.64/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal AddedInterface pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Add eth0 [10.132.0.117/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-be17e7-openshift-default Scaled up replica set e2e-worker-w6-be17e7-openshift-default-7c64556c75 from 2 to 3 51m Normal Pulled pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Container started 51m Normal Created pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Container created 51m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Readiness probe error: Get "http://10.133.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Readiness probe failed: Get "http://10.133.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Readiness probe failed: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Readiness probe error: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Readiness probe failed: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Readiness probe error: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Startup probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Startup probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 (combined from similar events): Created pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled up replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 6 to 10 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz 51m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Add eth0 [10.133.0.66/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 51m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Container created 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq 51m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js Add eth0 [10.132.0.125/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled up replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 2 to 4 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 51m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q Add eth0 [10.132.0.124/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr Add eth0 [10.134.0.63/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled up replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 4 to 8 51m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr Container created 51m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr Container started 51m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Container started 51m Normal Pulled pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Add eth0 [10.132.0.126/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Add eth0 [10.133.0.67/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz Add eth0 [10.134.0.64/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Add eth0 [10.133.0.68/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Container created 51m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Container started 51m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 Add eth0 [10.134.0.65/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 Container created 51m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz Container created 51m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Container created 51m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz Container started 51m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q Container created 51m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 Container started 51m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Container started 51m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js Container started 51m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Container started 51m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Container created 51m Normal Started pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q Container started 51m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Add eth0 [10.132.0.127/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Container created 51m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Container started 51m Normal Created pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js Container created 51m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled up replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 4 to 5 51m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Add eth0 [10.134.0.66/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled up replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 3 to 6 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd 51m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Container created 51m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Container created 51m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Container started 51m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Add eth0 [10.134.0.67/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Container created 51m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Container created 51m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Container started 51m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Container started 51m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Add eth0 [10.133.0.69/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Add eth0 [10.132.0.131/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Container started 51m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Startup probe failed: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Startup probe error: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 51m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-bdcn8 51m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-clhnh Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75bf867b6c from 6 to 4 51m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 51m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-bdcn8 Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-clhnh 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled up replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 3 to 6 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Created pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-be17e7-openshift-default Scaled up replica set e2e-worker-w6-be17e7-openshift-default-7c64556c75 from 3 to 6 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Created pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Created pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 50m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp Container created 50m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Container started 50m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Container started 50m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Container created 50m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Add eth0 [10.132.0.139/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp Container started 50m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp Add eth0 [10.134.0.69/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Container created 50m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 Container started 50m Normal Created pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 Container created 50m Normal Pulled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Add eth0 [10.133.0.71/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt Add eth0 [10.132.0.137/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 Add eth0 [10.133.0.70/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt Container created 50m Normal Started pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Container started 50m Normal Created pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Container created 50m Normal Pulled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Add eth0 [10.134.0.68/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt Container started 50m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt Startup probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Startup probe failed: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Startup probe error: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt Startup probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 7; reason: 50m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Readiness probe error: Get "http://10.132.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-be17e7-openshift-default Scaled up replica set e2e-worker-w6-be17e7-openshift-default-7c64556c75 from 6 to 7 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled up replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 6 to 7 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Created pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default New size: 7; reason: All metrics below target 50m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Readiness probe failed: Get "http://10.132.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj Add eth0 [10.134.0.70/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj Container started 50m Normal Pulled pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj Container started 50m Normal Created pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj Container created 50m Normal AddedInterface pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj Add eth0 [10.132.0.143/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj Container created 50m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 50m Normal AddedLabel gateway/e2e-ait-05e3625f Added label istio.io/rev=openshift-gateway to gateway e2e-ait-05e3625f 50m Normal ScalingReplicaSet deployment/e2e-ait-05e3625f-openshift-default Scaled up replica set e2e-ait-05e3625f-openshift-default-7fbdbcf75c from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-ait-05e3625f-openshift-default-7fbdbcf75c Created pod: e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h 50m Normal AddedInterface pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Add eth0 [10.132.0.153/23] from ovn-kubernetes 50m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-cv64p 50m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-cv64p Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75bf867b6c from 4 to 2 50m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-g7jrj Stopping container istio-proxy 50m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 50m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-g7jrj 50m Normal ScalingReplicaSet deployment/e2e-ait-ae20a62d-openshift-default Scaled up replica set e2e-ait-ae20a62d-openshift-default-77d6577fcf from 0 to 1 50m Normal AddedLabel gateway/e2e-ait-ae20a62d Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ae20a62d 50m Normal SuccessfulCreate replicaset/e2e-ait-ae20a62d-openshift-default-77d6577fcf Created pod: e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 50m Normal AddedInterface pod/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 Add eth0 [10.132.0.154/23] from ovn-kubernetes 50m Normal AddedLabel gateway/e2e-ait-dee83803 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-dee83803 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 50m Normal SuccessfulCreate replicaset/e2e-ait-dee83803-openshift-default-54775796bf Created pod: e2e-ait-dee83803-openshift-default-54775796bf-8vttp 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 50m Normal ScalingReplicaSet deployment/e2e-ait-dee83803-openshift-default Scaled up replica set e2e-ait-dee83803-openshift-default-54775796bf from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled up replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 5 to 9 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 (combined from similar events): Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 50m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Add eth0 [10.133.0.72/23] from ovn-kubernetes 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 50m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Add eth0 [10.132.0.156/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled up replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 8 to 10 50m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Add eth0 [10.133.0.73/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 (combined from similar events): Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c 50m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Add eth0 [10.133.0.74/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Created pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 50m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Container started 50m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Container created 50m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled up replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 6 to 10 50m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Created pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 50m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Add eth0 [10.132.0.158/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Container started 50m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-qjq6b 50m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Container created 50m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Add eth0 [10.134.0.71/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Add eth0 [10.134.0.74/23] from ovn-kubernetes 50m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 50m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Add eth0 [10.134.0.72/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Add eth0 [10.132.0.157/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 50m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Add eth0 [10.134.0.73/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Container created 50m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Container started 50m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Container created 50m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Container started 50m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Container started 50m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Container created 50m Normal Pulled pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Container created 50m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Container started 50m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Container created 50m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Container started 50m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Container created 50m Normal Started pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Container started 50m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Container created 50m Normal AddedInterface pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Add eth0 [10.133.0.75/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Container started 50m Normal Started pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Container started 50m Normal Created pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Container created 50m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Add eth0 [10.132.0.160/23] from ovn-kubernetes 50m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-05e3625f Scaled up replica set payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5 from 0 to 1 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-05e3625f Scaled up replica set payload-processing-e2e-ait-05e3625f-5c977df99 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5 Created pod: payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-gnzwz 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5 Error creating: pods "payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-05e3625f: serviceaccount "payload-processing-e2e-ait-05e3625f" not found 49m Warning FailedCreate replicaset/payload-processing-e2e-ait-05e3625f-5c977df99 Error creating: pods "payload-processing-e2e-ait-05e3625f-5c977df99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-05e3625f: serviceaccount "payload-processing-e2e-ait-05e3625f" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-05e3625f-5c977df99 Created pod: payload-processing-e2e-ait-05e3625f-5c977df99-tpplh 49m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Startup probe error: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Startup probe error: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Startup probe failed: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Startup probe failed: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Readiness probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Readiness probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Readiness probe failed: Get "http://10.132.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Readiness probe error: Get "http://10.132.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Startup probe error: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Startup probe failed: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused 49m Normal AddedInterface pod/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-gnzwz Add eth0 [10.133.0.76/23] from ovn-kubernetes 49m Normal AddedInterface pod/payload-processing-e2e-ait-05e3625f-5c977df99-tpplh Add eth0 [10.133.0.77/23] from ovn-kubernetes 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Readiness probe failed: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Pulled pod/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-gnzwz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Readiness probe error: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Startup probe error: Get "http://10.132.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal Pulled pod/payload-processing-e2e-ait-05e3625f-5c977df99-tpplh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Startup probe failed: Get "http://10.132.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Started pod/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-gnzwz Container started 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-ae20a62d Scaled up replica set payload-processing-e2e-ait-ae20a62d-76cbb85fc from 0 to 1 49m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Startup probe failed: Get "http://10.132.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Started pod/payload-processing-e2e-ait-05e3625f-5c977df99-tpplh Container started 49m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Created pod/payload-processing-e2e-ait-05e3625f-5c977df99-tpplh Container created 49m Normal Created pod/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-gnzwz Container created 49m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-ae20a62d Scaled up replica set payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5 from 0 to 1 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Startup probe error: HTTP probe failed with statuscode: 503... 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5 Created pod: payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-68fxm 49m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning FailedCreate replicaset/payload-processing-e2e-ait-ae20a62d-76cbb85fc Error creating: pods "payload-processing-e2e-ait-ae20a62d-76cbb85fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ae20a62d: serviceaccount "payload-processing-e2e-ait-ae20a62d" not found 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5 Error creating: pods "payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ae20a62d: serviceaccount "payload-processing-e2e-ait-ae20a62d" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-ae20a62d-76cbb85fc Created pod: payload-processing-e2e-ait-ae20a62d-76cbb85fc-2j2qs 49m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Startup probe error: Get "http://10.132.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Startup probe error: Get "http://10.132.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal Created pod/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-68fxm Container created 49m Normal Pulled pod/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-68fxm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-68fxm Add eth0 [10.133.0.78/23] from ovn-kubernetes 49m Normal Pulled pod/payload-processing-e2e-ait-ae20a62d-76cbb85fc-2j2qs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-ait-ae20a62d-76cbb85fc-2j2qs Container started 49m Normal Created pod/payload-processing-e2e-ait-ae20a62d-76cbb85fc-2j2qs Container created 49m Normal Started pod/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-68fxm Container started 49m Normal AddedInterface pod/payload-processing-e2e-ait-ae20a62d-76cbb85fc-2j2qs Add eth0 [10.133.0.79/23] from ovn-kubernetes 49m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 18.058s (18.058s including waiting). Image size: 185476477 bytes. 49m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Container started 49m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Container created 49m Warning FailedCreate replicaset/payload-processing-e2e-ait-dee83803-7f495bcb9f Error creating: pods "payload-processing-e2e-ait-dee83803-7f495bcb9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dee83803: serviceaccount "payload-processing-e2e-ait-dee83803" not found 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-dee83803 Scaled up replica set payload-processing-e2e-ait-dee83803-7f495bcb9f from 0 to 1 49m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Readiness probe error: HTTP probe failed with statuscode: 503... 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-dee83803 Scaled up replica set payload-pre-processing-e2e-ait-dee83803-cdc57876c from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-dee83803-cdc57876c Error creating: pods "payload-pre-processing-e2e-ait-dee83803-cdc57876c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dee83803: serviceaccount "payload-processing-e2e-ait-dee83803" not found 49m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Readiness probe failed: HTTP probe failed with statuscode: 503 49m Normal Created pod/payload-pre-processing-e2e-ait-dee83803-cdc57876c-xfwmv Container created 49m Normal Started pod/payload-pre-processing-e2e-ait-dee83803-cdc57876c-xfwmv Container started 49m Normal Pulled pod/payload-pre-processing-e2e-ait-dee83803-cdc57876c-xfwmv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-dee83803-cdc57876c Created pod: payload-pre-processing-e2e-ait-dee83803-cdc57876c-xfwmv 49m Normal AddedInterface pod/payload-pre-processing-e2e-ait-dee83803-cdc57876c-xfwmv Add eth0 [10.133.0.80/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-dee83803-7f495bcb9f Created pod: payload-processing-e2e-ait-dee83803-7f495bcb9f-xqbzg 49m Normal Pulled pod/payload-processing-e2e-ait-dee83803-7f495bcb9f-xqbzg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-ait-dee83803-7f495bcb9f-xqbzg Add eth0 [10.133.0.81/23] from ovn-kubernetes 49m Normal Started pod/payload-processing-e2e-ait-dee83803-7f495bcb9f-xqbzg Container started 49m Normal Created pod/payload-processing-e2e-ait-dee83803-7f495bcb9f-xqbzg Container created 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled up replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 7 to 9 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj 49m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Add eth0 [10.133.0.82/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Container created 49m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Container created 49m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Add eth0 [10.132.0.164/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Container started 49m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Container started 49m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Startup probe error: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Startup probe failed: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused 49m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled up replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 9 to 10 49m Normal Started pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht Container started 49m Normal AddedInterface pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht Add eth0 [10.134.0.76/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht Container created 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 (combined from similar events): Created pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f (combined from similar events): Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled up replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 9 to 10 49m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs Container created 49m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs Container started 49m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs Add eth0 [10.134.0.77/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75bf867b6c from 2 to 4 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-m55sg 48m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c Created pod: maas-default-gateway-openshift-default-75bf867b6c-hfhbd 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Add eth0 [10.133.0.84/23] from ovn-kubernetes 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-m55sg Add eth0 [10.133.0.83/23] from ovn-kubernetes 48m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-m55sg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Container created 48m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Container started 48m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-m55sg Container started 48m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-m55sg Container created 48m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 48m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-m55sg Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-m55sg Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ait-05e3625f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-ait-05e3625f-openshift-default Scaled up replica set e2e-ait-05e3625f-openshift-default-7fbdbcf75c from 1 to 4 48m Normal SuccessfulCreate replicaset/e2e-ait-05e3625f-openshift-default-7fbdbcf75c Created pod: e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j 48m Normal SuccessfulCreate replicaset/e2e-ait-05e3625f-openshift-default-7fbdbcf75c Created pod: e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 48m Normal SuccessfulCreate replicaset/e2e-ait-05e3625f-openshift-default-7fbdbcf75c Created pod: e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr 48m Normal Pulled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Container created 48m Normal Pulled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Container started 48m Normal Pulled pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Add eth0 [10.134.0.80/23] from ovn-kubernetes 48m Normal Started pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Container started 48m Normal Created pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Container created 48m Normal AddedInterface pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Add eth0 [10.132.0.165/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Add eth0 [10.133.0.85/23] from ovn-kubernetes 48m Normal Created pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Container created 48m Normal Created pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Container created 48m Normal Started pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Container started 48m Normal Started pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Container started 48m Normal Pulled pod/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 Container started 48m Normal Created pod/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 Container created 48m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Readiness probe failed: Get "http://10.132.0.153:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Readiness probe error: Get "http://10.132.0.153:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Started pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Container started 48m Normal Created pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Container created 48m Normal Pulled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning ProbeError pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 48m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 48m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Startup probe failed: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused 48m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Startup probe error: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused... 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dee83803-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dee83803-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 Startup probe error: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 Startup probe failed: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Startup probe failed: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused 48m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Startup probe error: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused... 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-05e3625f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-05e3625f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal Killing pod/payload-processing-e2e-ait-05e3625f-5c977df99-tpplh Stopping container payload-processing 48m Normal Killing pod/payload-pre-processing-e2e-ait-05e3625f-7488d6cbc5-gnzwz Stopping container payload-pre-processing 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-ae20a62d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-ae20a62d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal Killing pod/payload-processing-e2e-ait-ae20a62d-76cbb85fc-2j2qs Stopping container payload-processing 48m Normal Killing pod/payload-pre-processing-e2e-ait-ae20a62d-5c764c66f5-68fxm Stopping container payload-pre-processing 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-05e3625f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-05e3625f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-ae20a62d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-ae20a62d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal Killing pod/payload-pre-processing-e2e-ait-dee83803-cdc57876c-xfwmv Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-ait-dee83803-7f495bcb9f-xqbzg Stopping container payload-processing 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dee83803-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dee83803-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal SuccessfulCreate replicaset/e2e-ait-05e3625f-openshift-default-7fbdbcf75c Created pod: e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw 47m Normal SuccessfulCreate replicaset/e2e-ait-05e3625f-openshift-default-7fbdbcf75c Created pod: e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ait-05e3625f-openshift-default New size: 6; reason: All metrics below target 47m Normal ScalingReplicaSet deployment/e2e-ait-05e3625f-openshift-default Scaled up replica set e2e-ait-05e3625f-openshift-default-7fbdbcf75c from 4 to 6 47m Normal Started pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw Container started 47m Normal AddedInterface pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw Add eth0 [10.134.0.81/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw Container created 47m Normal Started pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz Container started 47m Normal Created pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz Container created 47m Normal Pulled pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz Add eth0 [10.133.0.86/23] from ovn-kubernetes 47m Normal Killing pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Stopping container istio-proxy 47m Normal Killing pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Stopping container istio-proxy 47m Normal Killing pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Stopping container istio-proxy 47m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 47m Normal Killing pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 47m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-dzmqw Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 47m Normal Killing pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-6h48h Stopping container istio-proxy 47m Normal Killing pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-4ggnz Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal SuccessfulCreate replicaset/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7 Created pod: e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc 47m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-js5jr Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-k5q29 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal AddedLabel gateway/e2e-migrate-cc8c7d75-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-cc8c7d75-gw 47m Normal ScalingReplicaSet deployment/e2e-migrate-cc8c7d75-gw-openshift-default Scaled up replica set e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7 from 0 to 1 47m Warning ProbeError pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal AddedInterface pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Add eth0 [10.133.0.87/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-ait-05e3625f-openshift-default-7fbdbcf75c-v2f2j Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-cc8c7d75 Scaled up replica set payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-cc8c7d75 Scaled up replica set payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687 from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c Error creating: pods "payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-cc8c7d75: serviceaccount "payload-processing-e2e-migrate-cc8c7d75" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c Created pod: payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-hwpsq 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687 Created pod: payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-6ntlk 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687 Error creating: pods "payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-cc8c7d75: serviceaccount "payload-processing-e2e-migrate-cc8c7d75" not found 47m Normal Created pod/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-6ntlk Container created 47m Normal Pulled pod/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-6ntlk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-6ntlk Add eth0 [10.133.0.88/23] from ovn-kubernetes 47m Normal AddedInterface pod/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-hwpsq Add eth0 [10.133.0.89/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-hwpsq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-hwpsq Container started 47m Normal Killing pod/e2e-ait-ae20a62d-openshift-default-77d6577fcf-qdsn6 Stopping container istio-proxy 47m Normal Created pod/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-hwpsq Container created 47m Normal Started pod/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-6ntlk Container started 47m Normal SuccessfulCreate replicaset/e2e-derive-e8ed1149-openshift-default-ff9d86fc Created pod: e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws 47m Normal AddedLabel gateway/e2e-derive-e8ed1149 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-e8ed1149 47m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75bf867b6c from 4 to 8 47m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/e2e-derive-e8ed1149-openshift-default Scaled up replica set e2e-derive-e8ed1149-openshift-default-ff9d86fc from 0 to 1 47m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Add eth0 [10.132.0.171/23] from ovn-kubernetes 47m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Container created 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Add eth0 [10.133.0.90/23] from ovn-kubernetes 47m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Container created 47m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Container started 47m Normal AddedInterface pod/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws Add eth0 [10.134.0.82/23] from ovn-kubernetes 47m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Container created 47m Normal AddedLabel gateway/e2e-shared-a-w2-926fc6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-926fc6 47m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Container started 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Add eth0 [10.134.0.84/23] from ovn-kubernetes 47m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Container started 47m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Container started 47m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Container created 47m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Add eth0 [10.134.0.83/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-926fc6-openshift-default Scaled up replica set e2e-shared-a-w2-926fc6-openshift-default-6b97494567 from 0 to 1 47m Normal Killing pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-926fc6-openshift-default-6b97494567 Created pod: e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 47m Normal AddedInterface pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Add eth0 [10.134.0.85/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-ait-dee83803-openshift-default-54775796bf-8vttp Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-e8ed1149-755778c777 Created pod: payload-processing-e2e-derive-e8ed1149-755778c777-nnmqv 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-e8ed1149 Scaled up replica set payload-processing-e2e-derive-e8ed1149-755778c777 from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-derive-e8ed1149-755778c777 Error creating: pods "payload-processing-e2e-derive-e8ed1149-755778c777-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-e8ed1149: serviceaccount "payload-processing-e2e-derive-e8ed1149" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5 Error creating: pods "payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-e8ed1149: serviceaccount "payload-processing-e2e-derive-e8ed1149" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5 Created pod: payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-ph6zx 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-e8ed1149 Scaled up replica set payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5 from 0 to 1 47m Normal Pulled pod/payload-processing-e2e-derive-e8ed1149-755778c777-nnmqv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-derive-e8ed1149-755778c777-nnmqv Container created 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal AddedInterface pod/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-ph6zx Add eth0 [10.132.0.172/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-ph6zx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-ph6zx Container created 47m Normal Started pod/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-ph6zx Container started 47m Normal AddedInterface pod/payload-processing-e2e-derive-e8ed1149-755778c777-nnmqv Add eth0 [10.132.0.173/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-derive-e8ed1149-755778c777-nnmqv Container started 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf Error creating: pods "payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-926fc6: serviceaccount "payload-processing-e2e-shared-a-w2-926fc6" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-926fc6 Scaled up replica set payload-processing-e2e-shared-a-w2-926fc6-7776b58857 from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-926fc6-7776b58857 Error creating: pods "payload-processing-e2e-shared-a-w2-926fc6-7776b58857-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-926fc6: serviceaccount "payload-processing-e2e-shared-a-w2-926fc6" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-926fc6 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-926fc6-7776b58857 Created pod: payload-processing-e2e-shared-a-w2-926fc6-7776b58857-8d7xh 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf Created pod: payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-27hbh 47m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-27hbh Container started 47m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-926fc6-7776b58857-8d7xh Add eth0 [10.132.0.175/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-shared-a-w2-926fc6-7776b58857-8d7xh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-27hbh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-27hbh Add eth0 [10.132.0.174/23] from ovn-kubernetes 47m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-27hbh Container created 47m Normal Started pod/payload-processing-e2e-shared-a-w2-926fc6-7776b58857-8d7xh Container started 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Created pod/payload-processing-e2e-shared-a-w2-926fc6-7776b58857-8d7xh Container created 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 47m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Container created 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Add eth0 [10.134.0.86/23] from ovn-kubernetes 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Add eth0 [10.132.0.176/23] from ovn-kubernetes 47m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Container started 47m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Container created 47m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Container started 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedLabel gateway/e2e-shared-a-w5-7c5d18 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-7c5d18 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7 Created pod: e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-7c5d18-openshift-default Scaled up replica set e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7 from 0 to 1 47m Normal AddedInterface pod/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm Add eth0 [10.134.0.87/23] from ovn-kubernetes 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Startup probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Startup probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 47m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-fxm8x 47m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 47m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Add eth0 [10.133.0.91/23] from ovn-kubernetes 47m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-7c5d18 Scaled up replica set payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9 from 0 to 1 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 47m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-7c5d18 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9 Created pod: payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-lz4ld 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5 Created pod: payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-vcbjl 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9 Error creating: pods "payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-7c5d18: serviceaccount "payload-processing-e2e-shared-a-w5-7c5d18" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-7c5d18: serviceaccount "payload-processing-e2e-shared-a-w5-7c5d18" not found 47m Normal Started pod/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-lz4ld Container started 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-vcbjl Add eth0 [10.132.0.177/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-vcbjl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-vcbjl Container created 47m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-vcbjl Container started 47m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-lz4ld Add eth0 [10.132.0.178/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-lz4ld Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-lz4ld Container created 46m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Container started 46m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.279s (4.279s including waiting). Image size: 185476477 bytes. 46m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Container created 46m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal AddedLabel gateway/e2e-mt-29bbe735 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-29bbe735 46m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr 46m Normal ScalingReplicaSet deployment/e2e-mt-29bbe735-openshift-default Scaled up replica set e2e-mt-29bbe735-openshift-default-cb78c776f from 0 to 1 46m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Add eth0 [10.132.0.179/23] from ovn-kubernetes 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-29bbe735-7d7b774575 Error creating: pods "payload-pre-processing-e2e-mt-29bbe735-7d7b774575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-29bbe735: serviceaccount "payload-processing-e2e-mt-29bbe735" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-29bbe735 Scaled up replica set payload-pre-processing-e2e-mt-29bbe735-7d7b774575 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-29bbe735-7d7b774575 Created pod: payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-29bbe735 Scaled up replica set payload-processing-e2e-mt-29bbe735-859dbf5fc from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-29bbe735-859dbf5fc Created pod: payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b 46m Warning FailedCreate replicaset/payload-processing-e2e-mt-29bbe735-859dbf5fc Error creating: pods "payload-processing-e2e-mt-29bbe735-859dbf5fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-29bbe735: serviceaccount "payload-processing-e2e-mt-29bbe735" not found 46m Normal Pulled pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn Container created 46m Normal AddedInterface pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b Add eth0 [10.132.0.181/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn Add eth0 [10.132.0.180/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b Container started 46m Normal Created pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b Container created 46m Normal SuccessfulCreate replicaset/e2e-mt-61346ee8-openshift-default-c79649cc Created pod: e2e-mt-61346ee8-openshift-default-c79649cc-r8659 46m Normal ScalingReplicaSet deployment/e2e-mt-61346ee8-openshift-default Scaled up replica set e2e-mt-61346ee8-openshift-default-c79649cc from 0 to 1 46m Normal AddedLabel gateway/e2e-mt-61346ee8 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-61346ee8 46m Normal AddedInterface pod/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 Add eth0 [10.132.0.182/23] from ovn-kubernetes 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-29bbe735-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-29bbe735-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 6; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled down replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 10 to 6 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Deleted pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Deleted pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Deleted pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Deleted pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-dj9js 46m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-v844q Stopping container istio-proxy 46m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-bcrmn Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Stopping container istio-proxy 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-61346ee8-7457f784f5 Created pod: payload-pre-processing-e2e-mt-61346ee8-7457f784f5-mcgws 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-61346ee8 Scaled up replica set payload-pre-processing-e2e-mt-61346ee8-7457f784f5 from 0 to 1 46m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-t792v Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-61346ee8-7457f784f5 Error creating: pods "payload-pre-processing-e2e-mt-61346ee8-7457f784f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-61346ee8: serviceaccount "payload-processing-e2e-mt-61346ee8" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-61346ee8 Scaled up replica set payload-processing-e2e-mt-61346ee8-789d6d5f75 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-61346ee8-789d6d5f75 Created pod: payload-processing-e2e-mt-61346ee8-789d6d5f75-xhp6q 46m Warning FailedCreate replicaset/payload-processing-e2e-mt-61346ee8-789d6d5f75 Error creating: pods "payload-processing-e2e-mt-61346ee8-789d6d5f75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-61346ee8: serviceaccount "payload-processing-e2e-mt-61346ee8" not found 46m Normal Created pod/payload-processing-e2e-mt-61346ee8-789d6d5f75-xhp6q Container created 46m Normal Started pod/payload-processing-e2e-mt-61346ee8-789d6d5f75-xhp6q Container started 46m Normal Pulled pod/payload-processing-e2e-mt-61346ee8-789d6d5f75-xhp6q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-mt-61346ee8-789d6d5f75-xhp6q Add eth0 [10.133.0.93/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-pre-processing-e2e-mt-61346ee8-7457f784f5-mcgws Add eth0 [10.133.0.92/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-mt-61346ee8-7457f784f5-mcgws Container created 46m Normal Started pod/payload-pre-processing-e2e-mt-61346ee8-7457f784f5-mcgws Container started 46m Normal Pulled pod/payload-pre-processing-e2e-mt-61346ee8-7457f784f5-mcgws Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Warning Unhealthy pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal Pulled pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Container created 46m Normal Started pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Container started 46m Normal Started pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Container started 46m Normal Created pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Container created 46m Normal Created pod/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws Container created 46m Normal Pulled pod/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws Container started 45m Warning ProbeError pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-926fc6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-926fc6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 45m Warning ProbeError pod/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-cc8c7d75-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-cc8c7d75-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-e8ed1149-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-e8ed1149-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/payload-pre-processing-e2e-migrate-cc8c7d75-59c6df8687-6ntlk Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-migrate-cc8c7d75-754bc6fc9c-hwpsq Stopping container payload-processing 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-e8ed1149-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-e8ed1149-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Started pod/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm Container started 45m Normal Pulled pod/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm Container created 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-926fc6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-926fc6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning ProbeError pod/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-cc8c7d75-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-cc8c7d75-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/payload-processing-e2e-derive-e8ed1149-755778c777-nnmqv Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-derive-e8ed1149-78cd6657f5-ph6zx Stopping container payload-pre-processing 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c Created pod: e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 45m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-5c2f68-openshift-default Scaled up replica set e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c from 0 to 1 45m Normal AddedLabel gateway/e2e-shared-b-w2-5c2f68 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-5c2f68 45m Normal AddedInterface pod/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 Add eth0 [10.132.0.185/23] from ovn-kubernetes 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-7c5d18-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-7c5d18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c Error creating: pods "payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-5c2f68: serviceaccount "payload-processing-e2e-shared-b-w2-5c2f68" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-5c2f68: serviceaccount "payload-processing-e2e-shared-b-w2-5c2f68" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-5c2f68 Scaled up replica set payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-5c2f68 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5 Created pod: payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c Created pod: payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-h6pqc 45m Normal Pulled pod/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-h6pqc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 Container started 45m Normal Created pod/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-h6pqc Container created 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 Add eth0 [10.132.0.186/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-h6pqc Container started 45m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-h6pqc Add eth0 [10.132.0.187/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 Container created 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Deleted pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Deleted pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 45m Normal Killing pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-kmnzj Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Readiness probe failed: Get "http://10.132.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Readiness probe error: Get "http://10.132.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Deleted pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt 45m Normal Killing pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-7xttt Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w6-be17e7-openshift-default Scaled down replica set e2e-worker-w6-be17e7-openshift-default-7c64556c75 from 7 to 4 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default New size: 4; reason: All metrics below target 45m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Container started 45m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Container created 45m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-pnvl9 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Startup probe error: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Startup probe failed: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-7c5d18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-7c5d18-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedLabel gateway/e2e-shared-b-w5-bbd10f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-bbd10f 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469 Created pod: e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-bbd10f-openshift-default Scaled up replica set e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469 from 0 to 1 44m Normal AddedInterface pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Add eth0 [10.132.0.188/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-bbd10f Scaled up replica set payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-bbd10f Scaled up replica set payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c Created pod: payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-v6zl6 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c Error creating: pods "payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-bbd10f: serviceaccount "payload-processing-e2e-shared-b-w5-bbd10f" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5 Created pod: payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-4fv8l 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5 Error creating: pods "payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-bbd10f: serviceaccount "payload-processing-e2e-shared-b-w5-bbd10f" not found 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-v6zl6 Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-v6zl6 Add eth0 [10.132.0.189/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-v6zl6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-4fv8l Container created 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-v6zl6 Container created 44m Normal Started pod/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-4fv8l Container started 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-4fv8l Add eth0 [10.132.0.190/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-4fv8l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Killing pod/e2e-migrate-cc8c7d75-gw-openshift-default-c77c885f7-5sjqc Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-71c235-openshift-default Scaled up replica set e2e-shared-a-w0-71c235-openshift-default-5668fb6485 from 0 to 1 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-29bbe735-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-29bbe735-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-71c235-openshift-default-5668fb6485 Created pod: e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs 44m Normal AddedLabel gateway/e2e-shared-a-w0-71c235 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-71c235 44m Normal AddedInterface pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Add eth0 [10.132.0.191/23] from ovn-kubernetes 44m Normal Killing pod/e2e-derive-e8ed1149-openshift-default-ff9d86fc-pccws Stopping container istio-proxy 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-73735c-openshift-default-f749cb549 Created pod: e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-73735c-openshift-default Scaled up replica set e2e-shared-a-w3-73735c-openshift-default-f749cb549 from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-a-w3-73735c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-73735c 44m Normal AddedInterface pod/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j Add eth0 [10.132.0.192/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-71c235 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-71c235 Scaled up replica set payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff Created pod: payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-5zmbn 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f Error creating: pods "payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-71c235: serviceaccount "payload-processing-e2e-shared-a-w0-71c235" not found 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff Error creating: pods "payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-71c235: serviceaccount "payload-processing-e2e-shared-a-w0-71c235" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f Created pod: payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-9z2d2 44m Normal Created pod/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-9z2d2 Container created 44m Normal Started pod/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-9z2d2 Container started 44m Normal AddedLabel gateway/e2e-api-a-a3600a Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-a3600a 44m Normal Pulled pod/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-9z2d2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-9z2d2 Add eth0 [10.132.0.193/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-5zmbn Container started 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-5zmbn Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-5zmbn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-5zmbn Add eth0 [10.132.0.194/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Add eth0 [10.132.0.195/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-api-a-a3600a-openshift-default-5d865f68dc Created pod: e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 44m Normal ScalingReplicaSet deployment/e2e-api-a-a3600a-openshift-default Scaled up replica set e2e-api-a-a3600a-openshift-default-5d865f68dc from 0 to 1 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 7; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Stopping container istio-proxy 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf Created pod: payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-fwdsz 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf Error creating: pods "payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-73735c: serviceaccount "payload-processing-e2e-shared-a-w3-73735c" not found 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff Error creating: pods "payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-73735c: serviceaccount "payload-processing-e2e-shared-a-w3-73735c" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff Created pod: payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-dh9xv 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-73735c Scaled up replica set payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff from 0 to 1 44m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-73735c Scaled up replica set payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf from 0 to 1 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 44m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled down replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 10 to 7 44m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 7; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-cl8gf Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled down replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 10 to 7 44m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-ptbht Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Deleted pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 44m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-rzws5 Stopping container istio-proxy 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-dh9xv Add eth0 [10.133.0.95/23] from ovn-kubernetes 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Deleted pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 44m Normal Pulled pod/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-dh9xv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj Stopping container istio-proxy 44m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-fwdsz Container started 44m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-qm5r5 Stopping container istio-proxy 44m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qjbc6 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-fwdsz Container created 44m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Deleted pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-rm7tj 44m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Deleted pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8h5jw 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Deleted pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-p4sbs 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Deleted pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mngqp 44m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 7; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz 44m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-gxq6c 44m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-hsk49 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default New size: 6; reason: All metrics below target 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 7; reason: All metrics below target 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled down replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 10 to 6 44m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled down replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 10 to 7 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-fwdsz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-d2rbz Stopping container istio-proxy 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-fwdsz Add eth0 [10.132.0.196/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled down replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 10 to 7 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Deleted pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb 44m Normal Started pod/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-dh9xv Container started 44m Normal Created pod/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-dh9xv Container created 44m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-267z6 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-8qfxb Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Readiness probe error: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-xfvw2 Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-5f9zs Readiness probe failed: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-a3600a-684485b7f9 Created pod: payload-pre-processing-e2e-api-a-a3600a-684485b7f9-7pnqr 44m Warning FailedCreate replicaset/payload-processing-e2e-api-a-a3600a-5767d5467c Error creating: pods "payload-processing-e2e-api-a-a3600a-5767d5467c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a3600a: serviceaccount "payload-processing-e2e-api-a-a3600a" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-a3600a Scaled up replica set payload-pre-processing-e2e-api-a-a3600a-684485b7f9 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-a3600a Scaled up replica set payload-processing-e2e-api-a-a3600a-5767d5467c from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-a3600a-5767d5467c Created pod: payload-processing-e2e-api-a-a3600a-5767d5467c-snzfh 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-a3600a-684485b7f9 Error creating: pods "payload-pre-processing-e2e-api-a-a3600a-684485b7f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a3600a: serviceaccount "payload-processing-e2e-api-a-a3600a" not found 44m Normal Created pod/payload-pre-processing-e2e-api-a-a3600a-684485b7f9-7pnqr Container created 44m Normal Pulled pod/payload-processing-e2e-api-a-a3600a-5767d5467c-snzfh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-a3600a-684485b7f9-7pnqr Add eth0 [10.132.0.197/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-api-a-a3600a-684485b7f9-7pnqr Container started 44m Normal Pulled pod/payload-pre-processing-e2e-api-a-a3600a-684485b7f9-7pnqr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-api-a-a3600a-5767d5467c-snzfh Add eth0 [10.132.0.198/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-api-a-a3600a-5767d5467c-snzfh Container created 44m Normal Started pod/payload-processing-e2e-api-a-a3600a-5767d5467c-snzfh Container started 43m Normal Created pod/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 Container created 43m Normal Pulled pod/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 Container started 43m Warning ProbeError pod/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 Startup probe error: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 Startup probe failed: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-61346ee8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-61346ee8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Pulled pod/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 Container created 43m Normal Started pod/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 Container started 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-5c2f68-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-5c2f68-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-61346ee8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-61346ee8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-5c2f68-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-5c2f68-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Pulled pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Container created 42m Normal Started pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Container started 42m Warning ProbeError pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Startup probe error: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Startup probe failed: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-71c235-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-71c235-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal AddedLabel gateway/e2e-api-a-721c75 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-721c75 42m Normal SuccessfulCreate replicaset/e2e-api-a-721c75-openshift-default-74d994d999 Created pod: e2e-api-a-721c75-openshift-default-74d994d999-g9nsl 42m Normal ScalingReplicaSet deployment/e2e-api-a-721c75-openshift-default Scaled up replica set e2e-api-a-721c75-openshift-default-74d994d999 from 0 to 1 42m Normal AddedInterface pod/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl Add eth0 [10.132.0.213/23] from ovn-kubernetes 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-71c235-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-71c235-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Killing pod/payload-processing-e2e-mt-61346ee8-789d6d5f75-xhp6q Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-mt-61346ee8-7457f784f5-mcgws Stopping container payload-pre-processing 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-721c75 Scaled up replica set payload-pre-processing-e2e-api-a-721c75-5f454d877f from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-721c75 Scaled up replica set payload-processing-e2e-api-a-721c75-6b765575f7 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-721c75-5f454d877f Error creating: pods "payload-pre-processing-e2e-api-a-721c75-5f454d877f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-721c75: serviceaccount "payload-processing-e2e-api-a-721c75" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-721c75-6b765575f7 Error creating: pods "payload-processing-e2e-api-a-721c75-6b765575f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-721c75: serviceaccount "payload-processing-e2e-api-a-721c75" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-721c75-5f454d877f Created pod: payload-pre-processing-e2e-api-a-721c75-5f454d877f-hr4x7 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-721c75-6b765575f7 Created pod: payload-processing-e2e-api-a-721c75-6b765575f7-zklj8 42m Normal AddedInterface pod/payload-processing-e2e-api-a-721c75-6b765575f7-zklj8 Add eth0 [10.132.0.214/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-api-a-721c75-6b765575f7-zklj8 Container started 42m Normal Created pod/payload-processing-e2e-api-a-721c75-6b765575f7-zklj8 Container created 42m Normal Pulled pod/payload-processing-e2e-api-a-721c75-6b765575f7-zklj8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-721c75-5f454d877f-hr4x7 Add eth0 [10.132.0.215/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-721c75-5f454d877f-hr4x7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-api-a-721c75-5f454d877f-hr4x7 Container created 42m Normal Started pod/payload-pre-processing-e2e-api-a-721c75-5f454d877f-hr4x7 Container started 42m Normal AddedLabel gateway/e2e-inf-a-c3c2cf Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c3c2cf 42m Normal ScalingReplicaSet deployment/e2e-inf-a-c3c2cf-openshift-default Scaled up replica set e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5 from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5 Created pod: e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx 42m Warning Unhealthy pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Readiness probe failed: Get "http://10.132.0.195:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Readiness probe error: Get "http://10.132.0.195:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal AddedInterface pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Add eth0 [10.132.0.216/23] from ovn-kubernetes 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 6; reason: All metrics below target 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 6; reason: All metrics below target 42m Warning ProbeError pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Readiness probe error: Get "http://10.132.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled down replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 7 to 6 42m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled down replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 7 to 6 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-8sf82 42m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-4kps7 42m Warning Unhealthy pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Readiness probe failed: Get "http://10.132.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 6; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled down replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 7 to 6 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bzv4d 42m Normal Started pod/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j Container started 42m Normal Pulled pod/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j Container created 42m Normal Started pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Container started 42m Normal Created pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Container created 42m Normal Pulled pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Container created 42m Normal Started pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Container started 42m Normal Pulled pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c3c2cf Scaled up replica set payload-processing-e2e-inf-a-c3c2cf-6887cd5b5 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7 Error creating: pods "payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c3c2cf: serviceaccount "payload-processing-e2e-inf-a-c3c2cf" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7 Created pod: payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-9jv96 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c3c2cf Scaled up replica set payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5 Created pod: payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-sghhb 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5 Error creating: pods "payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c3c2cf: serviceaccount "payload-processing-e2e-inf-a-c3c2cf" not found 41m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-v7krd 41m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-bdh5v 41m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-v7krd Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 41m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-bdh5v Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Startup probe failed: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused 41m Warning ProbeError pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Startup probe error: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused... 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-9jv96 Add eth0 [10.132.0.217/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j Startup probe failed: Get "http://10.132.0.192:15021/healthz/ready": dial tcp 10.132.0.192:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j Startup probe error: Get "http://10.132.0.192:15021/healthz/ready": dial tcp 10.132.0.192:15021: connect: connection refused... 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-sghhb Add eth0 [10.132.0.218/23] from ovn-kubernetes 41m Normal AddedLabel gateway/e2e-shared-b-w0-6116bd Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-6116bd 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-9jv96 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-a3600a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-a3600a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Created pod/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-9jv96 Container created 41m Normal Started pod/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-9jv96 Container started 41m Normal Pulled pod/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-sghhb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-6116bd-openshift-default Scaled up replica set e2e-shared-b-w0-6116bd-openshift-default-6c77499645 from 0 to 1 41m Normal Created pod/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-sghhb Container created 41m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-6116bd-openshift-default-6c77499645 Created pod: e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk 41m Normal Started pod/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-sghhb Container started 41m Warning Unhealthy pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 41m Normal AddedInterface pod/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk Add eth0 [10.132.0.219/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 41m Normal ScalingReplicaSet deployment/e2e-mt-29bbe735-openshift-default Scaled up replica set e2e-mt-29bbe735-openshift-default-cb78c776f from 1 to 2 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-bbd10f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-bbd10f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn 41m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn Add eth0 [10.134.0.99/23] from ovn-kubernetes 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-29bbe735-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn Container created 41m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn Container started 41m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 5; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default Scaled down replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 6 to 5 41m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Deleted pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mrtlr 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled down replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 6 to 5 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 5; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-9bkws 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled down replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 6 to 5 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-79vbw 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 5; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Deleted pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-49zd9 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 5; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-be17e7-openshift-default Scaled down replica set e2e-worker-w6-be17e7-openshift-default-7c64556c75 from 4 to 3 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default New size: 3; reason: All metrics below target 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default New size: 4; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled down replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 6 to 5 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled down replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 6 to 4 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Deleted pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Deleted pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-73735c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-73735c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tp9kf Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-6116bd Scaled up replica set payload-processing-e2e-shared-b-w0-6116bd-57d5945767 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987 Created pod: payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-72btg 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-6116bd-57d5945767 Created pod: payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm 41m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-6116bd-57d5945767 Error creating: pods "payload-processing-e2e-shared-b-w0-6116bd-57d5945767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-6116bd: serviceaccount "payload-processing-e2e-shared-b-w0-6116bd" not found 41m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-ljbn8 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-6116bd: serviceaccount "payload-processing-e2e-shared-b-w0-6116bd" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-6116bd Scaled up replica set payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987 from 0 to 1 41m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-9b6qv Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-72btg Container created 41m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-72btg Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-72btg Add eth0 [10.132.0.220/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm Add eth0 [10.134.0.100/23] from ovn-kubernetes 41m Normal AddedLabel gateway/e2e-shared-b-w3-d57980 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-d57980 41m Normal Pulling pod/payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 41m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-72btg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx Add eth0 [10.134.0.101/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-d57980-openshift-default-94f4f6945 Created pod: e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx 41m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-d57980-openshift-default Scaled up replica set e2e-shared-b-w3-d57980-openshift-default-94f4f6945 from 0 to 1 41m Normal Pulled pod/payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.246s (3.246s including waiting). Image size: 125353965 bytes. 41m Normal Started pod/payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm Container started 41m Normal Created pod/payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm Container created 41m Normal AddedLabel gateway/e2e-api-b-b5313f Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-b5313f 41m Normal ScalingReplicaSet deployment/e2e-api-b-b5313f-openshift-default Scaled up replica set e2e-api-b-b5313f-openshift-default-6986998b99 from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-api-b-b5313f-openshift-default-6986998b99 Created pod: e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx 41m Normal AddedInterface pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Add eth0 [10.134.0.102/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-d57980-644975995 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-d57980-644975995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d57980: serviceaccount "payload-processing-e2e-shared-b-w3-d57980" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-d57980 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-d57980-644975995 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-d57980 Scaled up replica set payload-processing-e2e-shared-b-w3-d57980-6948d6646c from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-d57980-6948d6646c Created pod: payload-processing-e2e-shared-b-w3-d57980-6948d6646c-2hpbb 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-d57980-6948d6646c Error creating: pods "payload-processing-e2e-shared-b-w3-d57980-6948d6646c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d57980: serviceaccount "payload-processing-e2e-shared-b-w3-d57980" not found 41m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-d57980-644975995-869mp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-d57980-644975995 Created pod: payload-pre-processing-e2e-shared-b-w3-d57980-644975995-869mp 41m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-d57980-644975995-869mp Container created 41m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-d57980-644975995-869mp Container started 41m Normal Created pod/payload-processing-e2e-shared-b-w3-d57980-6948d6646c-2hpbb Container created 41m Normal Started pod/payload-processing-e2e-shared-b-w3-d57980-6948d6646c-2hpbb Container started 41m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-d57980-6948d6646c-2hpbb Add eth0 [10.134.0.103/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-d57980-644975995-869mp Add eth0 [10.134.0.104/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-shared-b-w3-d57980-6948d6646c-2hpbb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-73735c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-73735c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-b5313f Scaled up replica set payload-processing-e2e-api-b-b5313f-685744b7c7 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-b5313f Scaled up replica set payload-pre-processing-e2e-api-b-b5313f-6679d85dcf from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-api-b-b5313f-685744b7c7 Error creating: pods "payload-processing-e2e-api-b-b5313f-685744b7c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-b5313f: serviceaccount "payload-processing-e2e-api-b-b5313f" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf Error creating: pods "payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-b5313f: serviceaccount "payload-processing-e2e-api-b-b5313f" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-b5313f-685744b7c7 Created pod: payload-processing-e2e-api-b-b5313f-685744b7c7-4lhg6 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf Created pod: payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-hfnzt 41m Normal Pulled pod/payload-processing-e2e-api-b-b5313f-685744b7c7-4lhg6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-hfnzt Container created 41m Normal Started pod/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-hfnzt Container started 41m Normal Pulled pod/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-hfnzt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-hfnzt Add eth0 [10.134.0.107/23] from ovn-kubernetes 41m Normal Created pod/payload-processing-e2e-api-b-b5313f-685744b7c7-4lhg6 Container created 41m Normal AddedInterface pod/payload-processing-e2e-api-b-b5313f-685744b7c7-4lhg6 Add eth0 [10.134.0.106/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-api-b-b5313f-685744b7c7-4lhg6 Container started 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-a3600a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-a3600a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-bbd10f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-bbd10f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Killing pod/e2e-mt-61346ee8-openshift-default-c79649cc-r8659 Stopping container istio-proxy 41m Normal Killing pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn Stopping container payload-pre-processing 40m Warning Unhealthy pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-l9d5b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning Unhealthy pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-g2xjn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-4xwjx Container started 40m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-4xwjx Container created 40m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Add eth0 [10.134.0.109/23] from ovn-kubernetes 40m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Container created 40m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Container started 40m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-4xwjx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-4xwjx Add eth0 [10.132.0.223/23] from ovn-kubernetes 40m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-4xwjx Startup probe error: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused... 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Startup probe failed: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-4xwjx Startup probe failed: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused 40m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Startup probe error: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused... 40m Normal Pulled pod/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl Container created 40m Normal Started pod/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl Container started 39m Warning Unhealthy pod/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl Startup probe failed: Get "http://10.132.0.213:15021/healthz/ready": dial tcp 10.132.0.213:15021: connect: connection refused 39m Warning ProbeError pod/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl Startup probe error: Get "http://10.132.0.213:15021/healthz/ready": dial tcp 10.132.0.213:15021: connect: connection refused... 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-721c75-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-721c75-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-721c75-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-721c75-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default Scaled down replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 7 to 5 39m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Deleted pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 5; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Deleted pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 39m Normal Pulled pod/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk Container created 39m Normal Started pod/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk Container started 39m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-88jz6 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Started pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Container started 39m Normal Created pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Container created 39m Normal Pulled pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-x2fct Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Started pod/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx Container started 39m Normal Created pod/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx Container created 39m Normal Pulled pod/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d57980-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d57980-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/payload-processing-e2e-api-a-721c75-6b765575f7-zklj8 Stopping container payload-processing 39m Warning ProbeError pod/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused... 39m Normal Killing pod/payload-pre-processing-e2e-api-a-721c75-5f454d877f-hr4x7 Stopping container payload-pre-processing 39m Warning Unhealthy pod/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused 39m Warning ProbeError pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-b5313f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-b5313f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Deleted pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 3; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default Scaled down replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 5 to 3 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Deleted pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl 39m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Stopping container istio-proxy 39m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default Scaled down replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 5 to 4 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 4; reason: All metrics below target 39m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-mfx7x Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-bg6pl Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-5rxnp Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal Killing pod/payload-processing-e2e-inf-a-c3c2cf-6887cd5b5-sghhb Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-inf-a-c3c2cf-7dc6ff5bb7-9jv96 Stopping container payload-pre-processing 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-6116bd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-6116bd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-b5313f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-b5313f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning ProbeError pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Readiness probe error: Get "http://10.132.0.216:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Readiness probe failed: Get "http://10.132.0.216:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal Pulled pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Container created 38m Normal Started pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Container started 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d57980-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d57980-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning ProbeError pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Startup probe error: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Startup probe failed: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-29bbe735 Scaled up replica set payload-processing-e2e-mt-29bbe735-859dbf5fc from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-29bbe735-7d7b774575 Created pod: payload-pre-processing-e2e-mt-29bbe735-7d7b774575-tclzz 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-29bbe735 Scaled up replica set payload-pre-processing-e2e-mt-29bbe735-7d7b774575 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-29bbe735-859dbf5fc Created pod: payload-processing-e2e-mt-29bbe735-859dbf5fc-ck47n 38m Warning FailedCreate replicaset/payload-processing-e2e-mt-29bbe735-859dbf5fc Error creating: pods "payload-processing-e2e-mt-29bbe735-859dbf5fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-29bbe735: serviceaccount "payload-processing-e2e-mt-29bbe735" not found 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-29bbe735-7d7b774575 Error creating: pods "payload-pre-processing-e2e-mt-29bbe735-7d7b774575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-29bbe735: serviceaccount "payload-processing-e2e-mt-29bbe735" not found 38m Normal Started pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-tclzz Container started 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-6116bd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal AddedInterface pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-ck47n Add eth0 [10.132.0.230/23] from ovn-kubernetes 38m Normal Pulled pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-ck47n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-ck47n Container created 38m Normal Started pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-ck47n Container started 38m Normal Created pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-tclzz Container created 38m Normal Pulled pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-tclzz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-tclzz Add eth0 [10.132.0.229/23] from ovn-kubernetes 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-6116bd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-71c235-55565ffff-5zmbn Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-shared-a-w0-71c235-64bb4ccc5f-9z2d2 Stopping container payload-processing 38m Normal Killing pod/payload-processing-e2e-shared-a-w3-73735c-5fc57bf7ff-dh9xv Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-73735c-55ff4fb7cf-fwdsz Stopping container payload-pre-processing 38m Normal Killing pod/payload-pre-processing-e2e-api-a-a3600a-684485b7f9-7pnqr Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-api-a-a3600a-5767d5467c-snzfh Stopping container payload-processing 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c3c2cf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c3c2cf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c3c2cf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c3c2cf-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal SuccessfulCreate replicaset/e2e-inf-a-d6b55f-openshift-default-68c88686bc Created pod: e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn 37m Normal ScalingReplicaSet deployment/e2e-inf-a-d6b55f-openshift-default Scaled up replica set e2e-inf-a-d6b55f-openshift-default-68c88686bc from 0 to 1 37m Normal AddedLabel gateway/e2e-inf-a-d6b55f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-d6b55f 37m Normal AddedInterface pod/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn Add eth0 [10.132.0.233/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-d6b55f Scaled up replica set payload-processing-e2e-inf-a-d6b55f-9d98cc4f from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-d6b55f-9d98cc4f Created pod: payload-processing-e2e-inf-a-d6b55f-9d98cc4f-dcmt4 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-d6b55f-9d98cc4f Error creating: pods "payload-processing-e2e-inf-a-d6b55f-9d98cc4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d6b55f: serviceaccount "payload-processing-e2e-inf-a-d6b55f" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-d6b55f Scaled up replica set payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5 Created pod: payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5 Error creating: pods "payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d6b55f: serviceaccount "payload-processing-e2e-inf-a-d6b55f" not found 37m Normal Created pod/payload-processing-e2e-inf-a-d6b55f-9d98cc4f-dcmt4 Container created 37m Normal Started pod/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs Container started 37m Normal Created pod/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs Container created 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs Add eth0 [10.132.0.234/23] from ovn-kubernetes 37m Normal Started pod/payload-processing-e2e-inf-a-d6b55f-9d98cc4f-dcmt4 Container started 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-d6b55f-9d98cc4f-dcmt4 Add eth0 [10.133.0.103/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-inf-a-d6b55f-9d98cc4f-dcmt4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Killing pod/e2e-api-a-721c75-openshift-default-74d994d999-g9nsl Stopping container istio-proxy 37m Normal SuccessfulCreate replicaset/e2e-inf-a-fb3406-openshift-default-75d976cfdf Created pod: e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq 37m Normal ScalingReplicaSet deployment/e2e-inf-a-fb3406-openshift-default Scaled up replica set e2e-inf-a-fb3406-openshift-default-75d976cfdf from 0 to 1 37m Normal AddedLabel gateway/e2e-inf-a-fb3406 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-fb3406 37m Normal AddedInterface pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Add eth0 [10.133.0.104/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-fb3406 Scaled up replica set payload-processing-e2e-inf-a-fb3406-896f479f9 from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-fb3406 Scaled up replica set payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5 from 0 to 1 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-fb3406-896f479f9 Error creating: pods "payload-processing-e2e-inf-a-fb3406-896f479f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-fb3406: serviceaccount "payload-processing-e2e-inf-a-fb3406" not found 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5 Error creating: pods "payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-fb3406: serviceaccount "payload-processing-e2e-inf-a-fb3406" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5 Created pod: payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-fb3406-896f479f9 Created pod: payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s 37m Normal Created pod/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl Container created 37m Normal Pulled pod/payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl Container started 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl Add eth0 [10.132.0.235/23] from ovn-kubernetes 37m Normal Started pod/payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s Container started 37m Normal Created pod/payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s Container created 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s Add eth0 [10.132.0.236/23] from ovn-kubernetes 37m Normal Killing pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-a-c3c2cf-openshift-default-bbfcd86f5-krchx Stopping container istio-proxy 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d6b55f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d6b55f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal Killing pod/e2e-shared-a-w3-73735c-openshift-default-f749cb549-gsq9j Stopping container istio-proxy 37m Warning ProbeError pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-shared-a-w0-71c235-openshift-default-5668fb6485-4rszs Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-6116bd-6448966987-72btg Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-shared-b-w0-6116bd-57d5945767-7jkvm Stopping container payload-processing 37m Normal Killing pod/payload-processing-e2e-shared-a-w5-7c5d18-57bb7d66f9-lz4ld Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-7c5d18-66dc74c5c5-vcbjl Stopping container payload-pre-processing 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq 37m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g 37m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-rv26g Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default New size: 3; reason: All metrics below target 37m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-d57980-644975995-869mp Stopping container payload-pre-processing 37m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Stopping container istio-proxy 37m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default New size: 4; reason: All metrics below target 37m Normal Killing pod/payload-processing-e2e-shared-b-w3-d57980-6948d6646c-2hpbb Stopping container payload-processing 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-f80a0e-openshift-default Scaled down replica set e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 from 4 to 3 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Deleted pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 37m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-be17e7-openshift-default-7c64556c75 Deleted pod: e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default New size: 3; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc 37m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f80a0e-openshift-default New size: 3; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5 Deleted pod: e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-be17e7-openshift-default Scaled down replica set e2e-worker-w6-be17e7-openshift-default-7c64556c75 from 3 to 2 37m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-q898z 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-be17e7-openshift-default New size: 2; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default Scaled down replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 5 to 4 37m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-fvlmc Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-tjcx6 Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-qmmqs Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-v9dj9 Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-lvzmq Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal Killing pod/e2e-shared-a-w5-7c5d18-openshift-default-5749d679f7-cqdnm Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-af30a9-openshift-default New size: 2; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 Deleted pod: e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf 36m Normal ScalingReplicaSet deployment/e2e-worker-w1-af30a9-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-af30a9-openshift-default-cbc4d6b59 from 3 to 2 36m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-fxm8x Stopping container discovery 36m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 36m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-fxm8x 36m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-7tpgf Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 36m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-bbd10f-6f97bbbb8c-v6zl6 Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-shared-b-w5-bbd10f-5f8cd8b7f5-4fv8l Stopping container payload-processing 36m Normal ScalingReplicaSet deployment/e2e-mt-29bbe735-openshift-default Scaled up replica set e2e-mt-29bbe735-openshift-default-cb78c776f from 2 to 4 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-29bbe735-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 36m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 36m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc 36m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 Container created 36m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc Add eth0 [10.132.0.241/23] from ovn-kubernetes 36m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc Container started 36m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 Container started 36m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc Container created 36m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 Add eth0 [10.133.0.107/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 Startup probe error: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc Startup probe failed: Get "http://10.132.0.241:15021/healthz/ready": dial tcp 10.132.0.241:15021: connect: connection refused 35m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc Startup probe error: Get "http://10.132.0.241:15021/healthz/ready": dial tcp 10.132.0.241:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 Startup probe failed: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-29bbe735-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 35m Normal ScalingReplicaSet deployment/e2e-mt-29bbe735-openshift-default Scaled up replica set e2e-mt-29bbe735-openshift-default-cb78c776f from 4 to 8 35m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p 35m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h 35m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc 35m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c 35m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h Add eth0 [10.132.0.244/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc Add eth0 [10.134.0.114/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c Container started 35m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p Add eth0 [10.134.0.113/23] from ovn-kubernetes 35m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc Container started 35m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc Container created 35m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p Container started 35m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c Container created 35m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c Add eth0 [10.133.0.108/23] from ovn-kubernetes 35m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p Container created 35m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Killing pod/e2e-shared-b-w0-6116bd-openshift-default-6c77499645-jnlgk Stopping container istio-proxy 35m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h Container created 35m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h Container started 35m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 35m Normal Killing pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Stopping container istio-proxy 35m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h Startup probe failed: Get "http://10.132.0.244:15021/healthz/ready": dial tcp 10.132.0.244:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 35m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc Startup probe failed: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused 35m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc Startup probe error: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h Startup probe error: Get "http://10.132.0.244:15021/healthz/ready": dial tcp 10.132.0.244:15021: connect: connection refused... 35m Normal Killing pod/e2e-shared-b-w3-d57980-openshift-default-94f4f6945-f2dxx Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-api-a-a3600a-openshift-default-5d865f68dc-fjcx6 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal AddedLabel gateway/e2e-api-a-771ff9 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-771ff9 35m Normal ScalingReplicaSet deployment/e2e-api-a-771ff9-openshift-default Scaled up replica set e2e-api-a-771ff9-openshift-default-7c4689f4d9 from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-api-a-771ff9-openshift-default-7c4689f4d9 Created pod: e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 35m Normal AddedInterface pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Add eth0 [10.132.0.246/23] from ovn-kubernetes 35m Normal Killing pod/payload-pre-processing-e2e-mt-29bbe735-7d7b774575-tclzz Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-mt-29bbe735-859dbf5fc-ck47n Stopping container payload-processing 35m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f (combined from similar events): Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg 35m Normal SuccessfulCreate replicaset/e2e-mt-29bbe735-openshift-default-cb78c776f Created pod: e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-29bbe735-openshift-default New size: 10; reason: 35m Normal ScalingReplicaSet deployment/e2e-mt-29bbe735-openshift-default Scaled up replica set e2e-mt-29bbe735-openshift-default-cb78c776f from 8 to 10 35m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg Add eth0 [10.132.0.247/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w Add eth0 [10.133.0.109/23] from ovn-kubernetes 35m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w Container created 35m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg Container created 35m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg Container started 35m Normal Started pod/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w Container started 35m Normal Pulled pod/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning FailedCreate replicaset/payload-processing-e2e-api-a-771ff9-65d554b547 Error creating: pods "payload-processing-e2e-api-a-771ff9-65d554b547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-771ff9: serviceaccount "payload-processing-e2e-api-a-771ff9" not found 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f Error creating: pods "payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-771ff9: serviceaccount "payload-processing-e2e-api-a-771ff9" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-771ff9 Scaled up replica set payload-processing-e2e-api-a-771ff9-65d554b547 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-771ff9 Scaled up replica set payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-771ff9-65d554b547 Created pod: payload-processing-e2e-api-a-771ff9-65d554b547-ct57x 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f Created pod: payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-qz5hj 35m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-qz5hj Add eth0 [10.132.0.249/23] from ovn-kubernetes 35m Normal Started pod/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-qz5hj Container started 35m Normal Pulled pod/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-qz5hj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-processing-e2e-api-a-771ff9-65d554b547-ct57x Container started 35m Normal Created pod/payload-processing-e2e-api-a-771ff9-65d554b547-ct57x Container created 35m Normal Pulled pod/payload-processing-e2e-api-a-771ff9-65d554b547-ct57x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-api-a-771ff9-65d554b547-ct57x Add eth0 [10.132.0.248/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-qz5hj Container created 35m Normal Killing pod/payload-processing-e2e-api-b-b5313f-685744b7c7-4lhg6 Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-api-b-b5313f-6679d85dcf-hfnzt Stopping container payload-pre-processing 35m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w Startup probe error: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg Startup probe failed: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w Startup probe failed: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused 35m Warning ProbeError pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg Startup probe error: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused... 35m Normal Pulled pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Container created 35m Normal Started pod/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn Container started 35m Normal Created pod/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn Container created 35m Normal Pulled pod/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Container started 35m Warning Unhealthy pod/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn Startup probe failed: Get "http://10.132.0.233:15021/healthz/ready": dial tcp 10.132.0.233:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn Startup probe error: Get "http://10.132.0.233:15021/healthz/ready": dial tcp 10.132.0.233:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Startup probe error: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Startup probe failed: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused 35m Normal Killing pod/e2e-shared-b-w5-bbd10f-openshift-default-6fd95cd469-vzr5q Stopping container istio-proxy 35m Normal AddedLabel gateway/e2e-inf-b-d585d2 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-d585d2 35m Normal ScalingReplicaSet deployment/e2e-inf-b-d585d2-openshift-default Scaled up replica set e2e-inf-b-d585d2-openshift-default-578cbd8b49 from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-inf-b-d585d2-openshift-default-578cbd8b49 Created pod: e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d6b55f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal AddedInterface pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Add eth0 [10.134.0.115/23] from ovn-kubernetes 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d6b55f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-d585d2 Scaled up replica set payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9 from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9 Error creating: pods "payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d585d2: serviceaccount "payload-processing-e2e-inf-b-d585d2" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-d585d2 Scaled up replica set payload-processing-e2e-inf-b-d585d2-747cfb8979 from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-d585d2-747cfb8979 Error creating: pods "payload-processing-e2e-inf-b-d585d2-747cfb8979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d585d2: serviceaccount "payload-processing-e2e-inf-b-d585d2" not found 34m Normal Pulled pod/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9 Created pod: payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm 34m Normal Started pod/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm Container started 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-d585d2-747cfb8979 Created pod: payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw 34m Normal Created pod/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm Container created 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm Add eth0 [10.132.0.251/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw Container started 34m Normal Created pod/payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw Container created 34m Normal AddedInterface pod/payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw Add eth0 [10.132.0.252/23] from ovn-kubernetes 34m Normal AddedLabel gateway/e2e-inf-b-afb98c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-afb98c 34m Normal Killing pod/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d Stopping container payload-processing 34m Normal SuccessfulCreate replicaset/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5 Created pod: e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv 34m Normal Killing pod/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k Stopping container payload-pre-processing 34m Normal ScalingReplicaSet deployment/e2e-inf-b-afb98c-openshift-default Scaled up replica set e2e-inf-b-afb98c-openshift-default-c8fb6f4c5 from 0 to 1 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-fb3406-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-fb3406-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal AddedInterface pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Add eth0 [10.132.0.253/23] from ovn-kubernetes 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-afb98c-fd7dd4465 Error creating: pods "payload-processing-e2e-inf-b-afb98c-fd7dd4465-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-afb98c: serviceaccount "payload-processing-e2e-inf-b-afb98c" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-afb98c Scaled up replica set payload-processing-e2e-inf-b-afb98c-fd7dd4465 from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-afb98c Scaled up replica set payload-pre-processing-e2e-inf-b-afb98c-76498858ff from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-afb98c-76498858ff Created pod: payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-afb98c-76498858ff Error creating: pods "payload-pre-processing-e2e-inf-b-afb98c-76498858ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-afb98c: serviceaccount "payload-processing-e2e-inf-b-afb98c" not found 34m Normal Started pod/payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw Container started 34m Normal Started pod/payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 Container started 34m Normal AddedInterface pod/payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw Add eth0 [10.133.0.111/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-afb98c-fd7dd4465 Created pod: payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw 34m Normal Created pod/payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 Container created 34m Normal Pulled pod/payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 Add eth0 [10.133.0.110/23] from ovn-kubernetes 34m Normal Created pod/payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw Container created 34m Normal Pulled pod/payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-f80a0e-7fbf86d675-7m46k Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-worker-w5-f80a0e-5bdbb549bf-jq24d Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Stopping container istio-proxy 34m Warning ProbeError pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-api-b-b5313f-openshift-default-6986998b99-kf6xx Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal Killing pod/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk Stopping container payload-pre-processing 33m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-af30a9-78db456cd7-7k9hk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-worker-w1-af30a9-b44c8fc5c-7jxcn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Readiness probe failed: Get "http://10.132.0.246:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Readiness probe error: Get "http://10.132.0.246:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Normal Created pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Container created 33m Normal Pulled pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Container started 33m Warning ProbeError pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Startup probe error: Get "http://10.132.0.246:15021/healthz/ready": dial tcp 10.132.0.246:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Startup probe failed: Get "http://10.132.0.246:15021/healthz/ready": dial tcp 10.132.0.246:15021: connect: connection refused 33m Normal Killing pod/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 Stopping container payload-processing 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-l597h 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-4xwjx 33m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Stopping container istio-proxy 33m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Stopping container istio-proxy 33m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-4xwjx Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-z7r5p Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ncnnc Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-jhf8w Stopping container istio-proxy 33m Normal SuccessfulCreate replicaset/e2e-api-b-9fce61-openshift-default-56798fcc9 Created pod: e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-ssfrg Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-np6gn Stopping container istio-proxy 33m Normal AddedLabel gateway/e2e-api-b-9fce61 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-9fce61 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-s6krr Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/e2e-api-b-9fce61-openshift-default Scaled up replica set e2e-api-b-9fce61-openshift-default-56798fcc9 from 0 to 1 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rjp6h Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-rqm74 Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-sj28c Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-29bbe735-openshift-default-cb78c776f-zgcwc Stopping container istio-proxy 33m Normal AddedInterface pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Add eth0 [10.134.0.119/23] from ovn-kubernetes 33m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-l597h Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-nwxll Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Readiness probe failed: Get "http://10.132.0.253:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning Unhealthy pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Readiness probe failed: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Readiness probe error: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning ProbeError pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Readiness probe error: Get "http://10.132.0.253:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-9fce61 Scaled up replica set payload-processing-e2e-api-b-9fce61-7bfc755f9c from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-9fce61 Scaled up replica set payload-pre-processing-e2e-api-b-9fce61-86b586b655 from 0 to 1 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-9fce61-86b586b655 Error creating: pods "payload-pre-processing-e2e-api-b-9fce61-86b586b655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-9fce61: serviceaccount "payload-processing-e2e-api-b-9fce61" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-9fce61-7bfc755f9c Created pod: payload-processing-e2e-api-b-9fce61-7bfc755f9c-vjgk6 33m Normal Pulled pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning FailedCreate replicaset/payload-processing-e2e-api-b-9fce61-7bfc755f9c Error creating: pods "payload-processing-e2e-api-b-9fce61-7bfc755f9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-9fce61: serviceaccount "payload-processing-e2e-api-b-9fce61" not found 33m Normal Started pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Container started 33m Normal Created pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Container created 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-9fce61-86b586b655 Created pod: payload-pre-processing-e2e-api-b-9fce61-86b586b655-x4pz8 33m Normal Created pod/payload-processing-e2e-api-b-9fce61-7bfc755f9c-vjgk6 Container created 33m Normal Started pod/payload-processing-e2e-api-b-9fce61-7bfc755f9c-vjgk6 Container started 33m Normal AddedInterface pod/payload-processing-e2e-api-b-9fce61-7bfc755f9c-vjgk6 Add eth0 [10.132.1.6/23] from ovn-kubernetes 33m Normal Pulled pod/payload-processing-e2e-api-b-9fce61-7bfc755f9c-vjgk6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Pulled pod/payload-pre-processing-e2e-api-b-9fce61-86b586b655-x4pz8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-9fce61-86b586b655-x4pz8 Add eth0 [10.132.1.7/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-api-b-9fce61-86b586b655-x4pz8 Container started 33m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-d2d4c3-5cb54fb9c9-vr6tb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal Created pod/payload-pre-processing-e2e-api-b-9fce61-86b586b655-x4pz8 Container created 33m Normal Pulled pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-771ff9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-771ff9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning Unhealthy pod/payload-processing-e2e-worker-w0-d2d4c3-847ddbf687-jwnm7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal Started pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Container started 33m Normal Created pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Container created 33m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-hnp7b Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-779lz Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Startup probe failed: Get "http://10.132.0.253:15021/healthz/ready": dial tcp 10.132.0.253:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Startup probe error: Get "http://10.132.0.253:15021/healthz/ready": dial tcp 10.132.0.253:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w5-f80a0e-openshift-default-59d68db9c5-q88n7 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-afb98c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-afb98c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d585d2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d585d2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-afb98c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-afb98c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d585d2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d585d2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Stopping container istio-proxy 32m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-rgzlh Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-worker-w1-af30a9-openshift-default-cbc4d6b59-fb64r Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-d2d4c3-openshift-default (combined from similar events): New size: 8; reason: cpu resource utilization (percentage of request) above target 32m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 Add eth0 [10.134.0.120/23] from ovn-kubernetes 32m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 Add eth0 [10.132.1.9/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 32m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 32m Normal SuccessfulCreate replicaset/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f Created pod: e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s 32m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn Add eth0 [10.132.1.8/23] from ovn-kubernetes 32m Normal AddedInterface pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s Add eth0 [10.134.0.121/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/e2e-worker-w0-d2d4c3-openshift-default (combined from similar events): Scaled up replica set e2e-worker-w0-d2d4c3-openshift-default-6567dd864f from 4 to 8 32m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 Container started 32m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s Container started 32m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 Container created 32m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn Container created 32m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 Container created 32m Normal Created pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s Container created 32m Normal Pulled pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn Container started 32m Normal Started pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 Container started 32m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 Startup probe failed: Get "http://10.132.1.9:15021/healthz/ready": dial tcp 10.132.1.9:15021: connect: connection refused 32m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn Startup probe error: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 Startup probe error: Get "http://10.132.1.9:15021/healthz/ready": dial tcp 10.132.1.9:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn Startup probe failed: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused 32m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s Startup probe failed: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused 32m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s Startup probe error: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused 32m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-9qn88 Container started 32m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-9qn88 Add eth0 [10.132.1.11/23] from ovn-kubernetes 32m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-zldcw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-zldcw Container created 32m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-zldcw Container started 32m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-zldcw Add eth0 [10.132.1.10/23] from ovn-kubernetes 32m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-9qn88 Container created 32m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-fj2bq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-9qn88 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-fj2bq Add eth0 [10.134.0.122/23] from ovn-kubernetes 32m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-fj2bq Container created 32m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-fj2bq Container started 32m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd 32m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-sfrjd Stopping container istio-proxy 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default New size: 3; reason: All metrics below target 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-fj2bq Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-9qn88 Startup probe failed: Get "http://10.132.1.11:15021/healthz/ready": dial tcp 10.132.1.11:15021: connect: connection refused 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-fj2bq Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-9qn88 Startup probe error: Get "http://10.132.1.11:15021/healthz/ready": dial tcp 10.132.1.11:15021: connect: connection refused... 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-zldcw Startup probe error: Get "http://10.132.1.10:15021/healthz/ready": dial tcp 10.132.1.10:15021: connect: connection refused... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-zldcw Startup probe failed: Get "http://10.132.1.10:15021/healthz/ready": dial tcp 10.132.1.10:15021: connect: connection refused 32m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qrmr7 Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-qmwq5 Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-5dsv4 Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-j6skn Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-b494s Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc 32m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 32m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-lwbgc Stopping container istio-proxy 32m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-624v6 Stopping container istio-proxy 32m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-mqrxj Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-bnpjq Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning Unhealthy pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w0-d2d4c3-openshift-default-6567dd864f-r7fhj Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal Pulled pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Container created 31m Normal Started pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Container started 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-9fce61-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-9fce61-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning ProbeError pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Startup probe error: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Startup probe failed: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-9fce61-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-9fce61-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal Killing pod/payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 Stopping container payload-pre-processing 30m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-be17e7-86c6ddb8dc-zcxv8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-processing-e2e-worker-w6-be17e7-546f498b5c-f5b7v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Killing pod/payload-pre-processing-e2e-api-a-771ff9-76cdd87b7f-qz5hj Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-api-a-771ff9-65d554b547-ct57x Stopping container payload-processing 30m Normal Killing pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-tkx26 Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w6-be17e7-openshift-default-7c64556c75-csxq4 Stopping container istio-proxy 29m Normal Killing pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-ldv4f 29m Warning Unhealthy pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-api-a-771ff9-openshift-default-7c4689f4d9-6xfp8 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal Killing pod/payload-processing-e2e-api-b-9fce61-7bfc755f9c-vjgk6 Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-api-b-9fce61-86b586b655-x4pz8 Stopping container payload-pre-processing 27m Normal ScalingReplicaSet deployment/e2e-worker-w4-b13755-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-b13755-openshift-default-656f8468c9 from 2 to 1 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-b13755-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 27m Normal SuccessfulDelete replicaset/e2e-worker-w4-b13755-openshift-default-656f8468c9 Deleted pod: e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 27m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 27m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 27m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-qjq6b Stopping container discovery 27m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-qjq6b 27m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-96t78 Stopping container istio-proxy 27m Normal SuccessfulDelete replicaset/e2e-worker-w3-23f3d4-openshift-default-54d69d7599 Deleted pod: e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m 27m Normal Killing pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Stopping container istio-proxy 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-23f3d4-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 27m Normal ScalingReplicaSet deployment/e2e-worker-w3-23f3d4-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-23f3d4-openshift-default-54d69d7599 from 2 to 1 27m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Stopping container istio-proxy 27m Warning ProbeError pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-api-b-9fce61-openshift-default-56798fcc9-c8n4q Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-8mh9m Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j Stopping container payload-pre-processing 27m Normal ScalingReplicaSet deployment/e2e-worker-w2-eb4aa1-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 from 2 to 1 27m Normal SuccessfulDelete replicaset/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945 Deleted pod: e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-eb4aa1-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 27m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-z8hw9 Stopping container istio-proxy 27m Normal Killing pod/payload-processing-e2e-inf-a-d6b55f-9d98cc4f-dcmt4 Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs Stopping container payload-pre-processing 27m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-23f3d4-564458bb8c-jq54j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-worker-w3-23f3d4-6b7474cb45-m9qzf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-inf-a-d6b55f-openshift-default-68c88686bc-bf4jn Stopping container istio-proxy 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-d6b55f-599cf486f5-kbgrs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Normal Killing pod/payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm Stopping container payload-pre-processing 26m Normal Killing pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Stopping container istio-proxy 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-d585d2-67f4bdbdf9-9vfbm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-processing-e2e-inf-b-d585d2-747cfb8979-7tgkw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning ProbeError pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-worker-w3-23f3d4-openshift-default-54d69d7599-p956m Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 26m Normal Killing pod/e2e-inf-b-d585d2-openshift-default-578cbd8b49-2qcxw Stopping container istio-proxy 26m Normal Killing pod/payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-worker-w4-b13755-557c67d777-rqfvg Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl Stopping container payload-pre-processing 25m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-fb3406-57867f4bc5-f6cnl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-processing-e2e-inf-a-fb3406-896f479f9-ptx5s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Stopping container istio-proxy 25m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-b13755-6cb84c449f-bplzl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning ProbeError pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-inf-a-fb3406-openshift-default-75d976cfdf-rgthq Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-m55sg Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75bf867b6c (combined from similar events): Deleted pod: maas-default-gateway-openshift-default-75bf867b6c-m55sg 25m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-9qn88 Stopping container istio-proxy 25m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Stopping container istio-proxy 25m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-fj2bq Stopping container istio-proxy 25m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-lhj8w Stopping container istio-proxy 25m Normal Killing pod/payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw Stopping container payload-processing 25m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-zldcw Stopping container istio-proxy 25m Normal Killing pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Stopping container istio-proxy 25m Normal Killing pod/payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 Stopping container payload-pre-processing 25m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-n42cb Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-zz7mk Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Stopping container istio-proxy 25m Normal Killing pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Stopping container istio-proxy 25m Warning ProbeError pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-inf-b-afb98c-openshift-default-c8fb6f4c5-qgslv Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/payload-processing-e2e-inf-b-afb98c-fd7dd4465-wd8dw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning ProbeError pod/e2e-worker-w4-b13755-openshift-default-656f8468c9-vdnqm Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-afb98c-76498858ff-f4646 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-926fc6-5c557ffccf-27hbh Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-a-w2-926fc6-7776b58857-8d7xh Stopping container payload-processing 24m Normal Killing pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Stopping container istio-proxy 24m Warning ProbeError pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-shared-a-w2-926fc6-openshift-default-6b97494567-znrf7 Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-shared-b-w2-5c2f68-5dbf8fbf6c-h6pqc Stopping container payload-processing 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-5c2f68-665485d7f5-rxz57 Readiness probe failed: timeout: failed to connect service "10.132.0.186:50052" within 1s: context deadline exceeded 24m Normal Killing pod/e2e-shared-b-w2-5c2f68-openshift-default-548b8f465c-spk77 Stopping container istio-proxy 24m Normal Killing pod/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 Stopping container payload-processing 23m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-eb4aa1-685ddbddc-splsl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-worker-w2-eb4aa1-76c5f496d7-jh5q2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Stopping container istio-proxy 23m Warning Unhealthy pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-worker-w2-eb4aa1-openshift-default-74c4b68945-qvjpm Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-5c66s Add eth0 [10.132.1.44/23] from ovn-kubernetes 22m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-5c66s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-cg4z5 Add eth0 [10.134.0.125/23] from ovn-kubernetes 22m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 6; reason: cpu resource utilization (percentage of request) above target 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-n92qw Add eth0 [10.134.0.126/23] from ovn-kubernetes 22m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-n92qw Container created 22m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-5c66s Container started 22m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-cg4z5 Container started 22m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-cg4z5 Container created 22m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-n92qw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-n92qw Container started 22m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-cg4z5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-5c66s Container created 22m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-cg4z5 Startup probe error: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-5c66s Startup probe failed: Get "http://10.132.1.44:15021/healthz/ready": dial tcp 10.132.1.44:15021: connect: connection refused 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-cg4z5 Startup probe failed: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-5c66s Startup probe error: Get "http://10.132.1.44:15021/healthz/ready": dial tcp 10.132.1.44:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-n92qw Startup probe failed: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-n92qw Startup probe error: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused... 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-75bf867b6c from 6 to 10 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75bf867b6c (combined from similar events): Created pod: maas-default-gateway-openshift-default-75bf867b6c-dgkjv 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 21m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-4kprw Container started 21m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-lxnwd Container started 21m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Container started 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-dgkjv Add eth0 [10.132.1.46/23] from ovn-kubernetes 21m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-dgkjv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Add eth0 [10.132.1.47/23] from ovn-kubernetes 21m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-dgkjv Container created 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-4kprw Add eth0 [10.134.0.128/23] from ovn-kubernetes 21m Normal Started pod/maas-default-gateway-openshift-default-75bf867b6c-dgkjv Container started 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-75bf867b6c-lxnwd Add eth0 [10.133.0.121/23] from ovn-kubernetes 21m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Container created 21m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-4kprw Container created 21m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-4kprw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Pulled pod/maas-default-gateway-openshift-default-75bf867b6c-lxnwd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-75bf867b6c-lxnwd Container created 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-4kprw Startup probe failed: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-lxnwd Startup probe error: Get "http://10.133.0.121:15021/healthz/ready": dial tcp 10.133.0.121:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-lxnwd Startup probe failed: Get "http://10.133.0.121:15021/healthz/ready": dial tcp 10.133.0.121:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-4kprw Startup probe error: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-dgkjv Startup probe failed: Get "http://10.132.1.46:15021/healthz/ready": dial tcp 10.132.1.46:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-dgkjv Startup probe error: Get "http://10.132.1.46:15021/healthz/ready": dial tcp 10.132.1.46:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Startup probe failed: Get "http://10.132.1.47:15021/healthz/ready": dial tcp 10.132.1.47:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Startup probe error: Get "http://10.132.1.47:15021/healthz/ready": dial tcp 10.132.1.47:15021: connect: connection refused... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Readiness probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-wgcfm Readiness probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Readiness probe error: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-hfhbd Readiness probe failed: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Normal SuccessfulCreate replicaset/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc Created pod: e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k 15m Normal ScalingReplicaSet deployment/e2e-worker-single-main-dcc58b-openshift-default Scaled up replica set e2e-worker-single-main-dcc58b-openshift-default-75b6945cc from 0 to 1 15m Normal AddedLabel gateway/e2e-worker-single-main-dcc58b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-dcc58b 15m Normal AddedInterface pod/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k Add eth0 [10.134.0.129/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-dcc58b-5487f988d5 Created pod: payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr 15m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-dcc58b-5487f988d5 Error creating: pods "payload-processing-e2e-worker-single-main-dcc58b-5487f988d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-dcc58b: serviceaccount "payload-processing-e2e-worker-single-main-dcc58b" not found 15m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75cb48c Error creating: pods "payload-pre-processing-e2e-worker-single-main-dcc58b-69b75cb48c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-dcc58b: serviceaccount "payload-processing-e2e-worker-single-main-dcc58b" not found 15m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75cb48c Created pod: payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml 15m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-dcc58b Scaled up replica set payload-processing-e2e-worker-single-main-dcc58b-5487f988d5 from 0 to 1 15m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-dcc58b Scaled up replica set payload-pre-processing-e2e-worker-single-main-dcc58b-69b75cb48c from 0 to 1 15m Normal Created pod/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml Container created 15m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml Add eth0 [10.132.1.61/23] from ovn-kubernetes 15m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr Add eth0 [10.134.0.130/23] from ovn-kubernetes 15m Normal Started pod/payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr Container started 15m Normal Pulled pod/payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Started pod/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml Container started 15m Normal Created pod/payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr Container created 15m Normal Pulled pod/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal Created pod/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k Container created 15m Normal Started pod/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k Container started 15m Warning Unhealthy pod/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k Startup probe failed: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused 15m Warning ProbeError pod/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k Startup probe error: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused... 14m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-dcc58b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 14m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-dcc58b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 14m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-dcc58b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-dcc58b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr Stopping container payload-processing 13m Normal Killing pod/e2e-worker-single-main-dcc58b-openshift-default-75b6945cc-tk98k Stopping container istio-proxy 13m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-dcc58b-5487f988dsqmrr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-dcc58b-69b75jqmml Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-2e092e-openshift-default-86469f4d57 Created pod: e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 11m Normal AddedLabel gateway/e2e-worker-single-main-2e092e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-2e092e 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-2e092e-openshift-default Scaled up replica set e2e-worker-single-main-2e092e-openshift-default-86469f4d57 from 0 to 1 11m Normal AddedInterface pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Add eth0 [10.134.0.132/23] from ovn-kubernetes 10m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-2e092e Scaled up replica set payload-processing-e2e-worker-single-main-2e092e-57799b8f49 from 0 to 1 10m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-2e092e Scaled up replica set payload-pre-processing-e2e-worker-single-main-2e092e-78f67db8c5 from 0 to 1 10m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-2e092e-78f67db8c5 Created pod: payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl 10m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-2e092e-78f67db8c5 Error creating: pods "payload-pre-processing-e2e-worker-single-main-2e092e-78f67db8c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2e092e: serviceaccount "payload-processing-e2e-worker-single-main-2e092e" not found 10m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-2e092e-57799b8f49 Created pod: payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r 10m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-2e092e-57799b8f49 Error creating: pods "payload-processing-e2e-worker-single-main-2e092e-57799b8f49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2e092e: serviceaccount "payload-processing-e2e-worker-single-main-2e092e" not found 10m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl Add eth0 [10.132.1.69/23] from ovn-kubernetes 10m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r Add eth0 [10.134.0.133/23] from ovn-kubernetes 10m Normal Started pod/payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r Container started 10m Normal Pulled pod/payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Started pod/payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl Container started 10m Normal Created pod/payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r Container created 10m Normal Created pod/payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl Container created 10m Normal Started pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Container started 10m Normal Pulled pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Normal Created pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Container created 10m Warning ProbeError pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Startup probe error: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused... 10m Warning Unhealthy pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Startup probe failed: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-2e092e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-2e092e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m18s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl Stopping container payload-pre-processing 9m18s Normal Killing pod/payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r Stopping container payload-processing 9m1s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-2e092e-78f674nhnl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 9m1s Normal Killing pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Stopping container istio-proxy 9m1s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-2e092e-57799b8f4rl29r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m59s Warning Unhealthy pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Readiness probe failed: HTTP probe failed with statuscode: 503 8m59s Warning ProbeError pod/e2e-worker-single-main-2e092e-openshift-default-86469f4d57n95q8 Readiness probe error: HTTP probe failed with statuscode: 503... 8m2s Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Readiness probe failed: Get "http://10.133.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m2s Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-nr4sp Readiness probe error: Get "http://10.133.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m27s Normal ScalingReplicaSet deployment/e2e-ait-6e41e55b-openshift-default Scaled up replica set e2e-ait-6e41e55b-openshift-default-789db867c9 from 0 to 1 6m27s Normal AddedLabel gateway/e2e-ait-6e41e55b Added label istio.io/rev=openshift-gateway to gateway e2e-ait-6e41e55b 6m27s Normal SuccessfulCreate replicaset/e2e-ait-6e41e55b-openshift-default-789db867c9 Created pod: e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn 6m26s Normal AddedInterface pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Add eth0 [10.132.1.75/23] from ovn-kubernetes 6m20s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-6e41e55b Scaled up replica set payload-processing-e2e-ait-6e41e55b-77fbff758f from 0 to 1 6m20s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f Error creating: pods "payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-6e41e55b: serviceaccount "payload-processing-e2e-ait-6e41e55b" not found 6m20s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-6e41e55b Scaled up replica set payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f from 0 to 1 6m20s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-6e41e55b-77fbff758f Created pod: payload-processing-e2e-ait-6e41e55b-77fbff758f-7xjhc 6m20s Warning FailedCreate replicaset/payload-processing-e2e-ait-6e41e55b-77fbff758f Error creating: pods "payload-processing-e2e-ait-6e41e55b-77fbff758f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-6e41e55b: serviceaccount "payload-processing-e2e-ait-6e41e55b" not found 6m19s Normal Created pod/payload-processing-e2e-ait-6e41e55b-77fbff758f-7xjhc Container created 6m19s Normal Started pod/payload-processing-e2e-ait-6e41e55b-77fbff758f-7xjhc Container started 6m19s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f Created pod: payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-nhrtj 6m19s Normal Pulled pod/payload-processing-e2e-ait-6e41e55b-77fbff758f-7xjhc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m19s Normal AddedInterface pod/payload-processing-e2e-ait-6e41e55b-77fbff758f-7xjhc Add eth0 [10.132.1.76/23] from ovn-kubernetes 6m19s Normal Pulled pod/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-nhrtj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m19s Normal AddedInterface pod/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-nhrtj Add eth0 [10.134.0.138/23] from ovn-kubernetes 6m18s Normal Started pod/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-nhrtj Container started 6m18s Normal Created pod/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-nhrtj Container created 6m12s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-6e41e55b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 6m12s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-6e41e55b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 6m2s Normal Started pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Container started 6m2s Normal Created pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Container created 6m2s Normal Pulled pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m57s Warning Unhealthy pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Startup probe failed: Get "http://10.132.1.75:15021/healthz/ready": dial tcp 10.132.1.75:15021: connect: connection refused 5m57s Warning ProbeError pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Startup probe error: Get "http://10.132.1.75:15021/healthz/ready": dial tcp 10.132.1.75:15021: connect: connection refused... 5m42s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-6e41e55b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m42s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-6e41e55b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m40s Normal Killing pod/payload-processing-e2e-ait-6e41e55b-77fbff758f-7xjhc Stopping container payload-processing 5m40s Normal Killing pod/payload-pre-processing-e2e-ait-6e41e55b-7fc48d875f-nhrtj Stopping container payload-pre-processing 5m31s Normal Killing pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Stopping container istio-proxy 5m29s Normal AddedLabel gateway/e2e-authcfg-http-152666 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-152666 5m29s Normal ScalingReplicaSet deployment/e2e-authcfg-http-152666-openshift-default Scaled up replica set e2e-authcfg-http-152666-openshift-default-7bc87b8cbf from 0 to 1 5m29s Normal SuccessfulCreate replicaset/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf Created pod: e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 5m28s Normal AddedInterface pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Add eth0 [10.134.0.139/23] from ovn-kubernetes 5m26s Warning Unhealthy pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Readiness probe failed: HTTP probe failed with statuscode: 503 5m26s Warning ProbeError pod/e2e-ait-6e41e55b-openshift-default-789db867c9-x2kzn Readiness probe error: HTTP probe failed with statuscode: 503... 5m21s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-152666-5b4bb5865f Error creating: pods "payload-processing-e2e-authcfg-http-152666-5b4bb5865f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-152666: serviceaccount "payload-processing-e2e-authcfg-http-152666" not found 5m21s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-152666-5b4bb5865f Created pod: payload-processing-e2e-authcfg-http-152666-5b4bb5865f-2qjwp 5m21s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-152666 Scaled up replica set payload-processing-e2e-authcfg-http-152666-5b4bb5865f from 0 to 1 5m21s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7 Error creating: pods "payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-152666: serviceaccount "payload-processing-e2e-authcfg-http-152666" not found 5m21s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7 Created pod: payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-b7ps4 5m21s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-152666 Scaled up replica set payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7 from 0 to 1 5m20s Normal Started pod/payload-processing-e2e-authcfg-http-152666-5b4bb5865f-2qjwp Container started 5m20s Normal Pulled pod/payload-processing-e2e-authcfg-http-152666-5b4bb5865f-2qjwp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m20s Normal Started pod/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-b7ps4 Container started 5m20s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-152666-5b4bb5865f-2qjwp Add eth0 [10.132.1.79/23] from ovn-kubernetes 5m20s Normal Created pod/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-b7ps4 Container created 5m20s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-b7ps4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m20s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-b7ps4 Add eth0 [10.132.1.78/23] from ovn-kubernetes 5m20s Normal Created pod/payload-processing-e2e-authcfg-http-152666-5b4bb5865f-2qjwp Container created 5m1s Normal Created pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Container created 5m1s Normal Started pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Container started 5m1s Normal Pulled pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m59s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-152666-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m59s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-152666-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m56s Warning ProbeError pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused... 4m56s Warning Unhealthy pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused 4m28s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-152666-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m28s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-152666-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m27s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-152666-66b4cc55d7-b7ps4 Stopping container payload-pre-processing 3m27s Normal Killing pod/payload-processing-e2e-authcfg-http-152666-5b4bb5865f-2qjwp Stopping container payload-processing 3m10s Warning ProbeError pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Readiness probe error: HTTP probe failed with statuscode: 503... 3m10s Normal Killing pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Stopping container istio-proxy 3m10s Warning Unhealthy pod/e2e-authcfg-http-152666-openshift-default-7bc87b8cbf-rdvn8 Readiness probe failed: HTTP probe failed with statuscode: 503 111s Warning ProbeError pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Readiness probe error: Get "http://10.132.1.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 111s Warning Unhealthy pod/maas-default-gateway-openshift-default-75bf867b6c-rwbtf Readiness probe failed: Get "http://10.132.1.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)