=== events in opendatahub (2026-10-30T17:38:08Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 72m Normal Scheduled pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Successfully assigned opendatahub/opendatahub-operator-controller-manager-cbcd98d6d-l9znm to ip-10-0-131-136.ec2.internal 71m Normal Scheduled pod/kserve-controller-manager-776cc66d47-vg6k7 Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-vg6k7 to ip-10-0-128-181.ec2.internal 51m Normal Scheduled pod/test-curl-8968-1aa1d0 Successfully assigned opendatahub/test-curl-8968-1aa1d0 to ip-10-0-131-136.ec2.internal 57m Normal Scheduled pod/test-curl-8965-cbe3f3 Successfully assigned opendatahub/test-curl-8965-cbe3f3 to ip-10-0-131-136.ec2.internal 57m Normal Scheduled pod/test-curl-8965-9edf49 Successfully assigned opendatahub/test-curl-8965-9edf49 to ip-10-0-131-136.ec2.internal 47m Normal Scheduled pod/test-curl-8962-44aa17 Successfully assigned opendatahub/test-curl-8962-44aa17 to ip-10-0-128-181.ec2.internal 51m Normal Scheduled pod/test-curl-8962-2f0f36 Successfully assigned opendatahub/test-curl-8962-2f0f36 to ip-10-0-131-136.ec2.internal 36m Normal Scheduled pod/test-curl-8959-424b98 Successfully assigned opendatahub/test-curl-8959-424b98 to ip-10-0-128-181.ec2.internal 51m Normal Scheduled pod/test-curl-8968-42007b Successfully assigned opendatahub/test-curl-8968-42007b to ip-10-0-131-136.ec2.internal 33m Normal Scheduled pod/test-curl-8956-25e9f6 Successfully assigned opendatahub/test-curl-8956-25e9f6 to ip-10-0-128-181.ec2.internal 65m Normal Scheduled pod/ai-gateway-controller-6d48fc876f-4g6d7 Successfully assigned opendatahub/ai-gateway-controller-6d48fc876f-4g6d7 to ip-10-0-129-65.ec2.internal 51m Normal Scheduled pod/test-curl-8968-efdfc3 Successfully assigned opendatahub/test-curl-8968-efdfc3 to ip-10-0-131-136.ec2.internal 33m Normal Scheduled pod/test-curl-8956-01d266 Successfully assigned opendatahub/test-curl-8956-01d266 to ip-10-0-128-181.ec2.internal 67m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-vt287 Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-vt287 to ip-10-0-128-181.ec2.internal 19m Normal Scheduled pod/maas-controller-575c7b5c5b-f64sn Successfully assigned opendatahub/maas-controller-575c7b5c5b-f64sn to ip-10-0-131-136.ec2.internal 67m Normal Scheduled pod/maas-controller-575c7b5c5b-lh44m Successfully assigned opendatahub/maas-controller-575c7b5c5b-lh44m to ip-10-0-131-136.ec2.internal 18m Normal Scheduled pod/maas-controller-575c7b5c5b-p8bxg Successfully assigned opendatahub/maas-controller-575c7b5c5b-p8bxg to ip-10-0-131-136.ec2.internal 67m Normal Scheduled pod/usage-logs-tenancy-proxy-7d94f55f4b-4f74l Successfully assigned opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-4f74l to ip-10-0-131-136.ec2.internal 51m Normal Scheduled pod/test-curl-8968-f1788c Successfully assigned opendatahub/test-curl-8968-f1788c to ip-10-0-131-136.ec2.internal 67m Normal Scheduled pod/maas-controller-577f596567-h5692 Successfully assigned opendatahub/maas-controller-577f596567-h5692 to ip-10-0-131-136.ec2.internal 65m Normal Scheduled pod/ai-gateway-controller-6cf7bcbfb6-s4pl8 Successfully assigned opendatahub/ai-gateway-controller-6cf7bcbfb6-s4pl8 to ip-10-0-129-65.ec2.internal 71m Normal Scheduled pod/model-serving-api-75bd89fdfc-zgr4b Successfully assigned opendatahub/model-serving-api-75bd89fdfc-zgr4b to ip-10-0-128-181.ec2.internal 71m Normal Scheduled pod/odh-model-controller-f87fcf56c-5fmhl Successfully assigned opendatahub/odh-model-controller-f87fcf56c-5fmhl to ip-10-0-128-181.ec2.internal 72m Normal Scheduled pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Successfully assigned opendatahub/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw to ip-10-0-129-65.ec2.internal 72m Normal Scheduled pod/odh-observability-584f99c788-k7579 Successfully assigned opendatahub/odh-observability-584f99c788-k7579 to ip-10-0-128-181.ec2.internal 72m Normal Scheduled pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Successfully assigned opendatahub/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn to ip-10-0-128-181.ec2.internal 72m Normal Scheduled pod/kserve-module-controller-manager-5cfb77bf8c-hv6k2 Successfully assigned opendatahub/kserve-module-controller-manager-5cfb77bf8c-hv6k2 to ip-10-0-131-136.ec2.internal 71m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-nxcfv Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-nxcfv to ip-10-0-128-181.ec2.internal 72m Normal RequirementsUnknown clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 requirements not yet checked 72m Normal ScalingReplicaSet deployment/opendatahub-operator-controller-manager Scaled up replica set opendatahub-operator-controller-manager-cbcd98d6d from 0 to 3 72m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-cbcd98d6d Created pod: opendatahub-operator-controller-manager-cbcd98d6d-xbsxw 72m Normal AllRequirementsMet clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 all requirements found, attempting install 72m Normal RequirementsNotMet clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 one or more requirements couldn't be found 72m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-cbcd98d6d Created pod: opendatahub-operator-controller-manager-cbcd98d6d-l9znm 72m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-cbcd98d6d Created pod: opendatahub-operator-controller-manager-cbcd98d6d-xpdtn 72m Normal Pulling pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal AddedInterface pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Add eth0 [10.132.0.32/23] from ovn-kubernetes 72m Normal Pulling pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal AddedInterface pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Add eth0 [10.133.0.16/23] from ovn-kubernetes 72m Normal AddedInterface pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Add eth0 [10.134.0.27/23] from ovn-kubernetes 72m Normal Pulling pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal InstallSucceeded clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 waiting for install components to report healthy 72m Normal InstallWaiting clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability. 72m Normal Pulled pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.784s (2.784s including waiting). Image size: 273709574 bytes. 72m Normal Pulling pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal Pulled pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.817s (2.817s including waiting). Image size: 273709574 bytes. 72m Normal Started pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Container started 72m Normal Created pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Container created 72m Normal Created pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Container created 72m Normal Started pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Container started 72m Normal Created pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Container created 72m Normal Started pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Container started 72m Normal Pulled pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.789s (2.789s including waiting). Image size: 273709574 bytes. 72m Normal Created pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Container created 72m Normal Pulled pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 363ms (363ms including waiting). Image size: 273709574 bytes. 72m Normal Pulling pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal Created pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Container created 72m Normal Started pod/opendatahub-operator-controller-manager-cbcd98d6d-xpdtn Container started 72m Normal LeaderElection lease/07ed84f7.opendatahub.io opendatahub-operator-controller-manager-cbcd98d6d-xpdtn_d8c167ea-ce8f-4e02-93ee-d968c406dab2 became leader 72m Normal Pulling pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal Pulled pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 453ms (453ms including waiting). Image size: 273709574 bytes. 72m Normal Started pod/opendatahub-operator-controller-manager-cbcd98d6d-xbsxw Container started 72m Normal Created pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Container created 72m Normal Started pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Container started 72m Normal Pulled pod/opendatahub-operator-controller-manager-cbcd98d6d-l9znm Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 303ms (303ms including waiting). Image size: 273709574 bytes. 72m Normal InstallSucceeded clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 install strategy completed with no errors 72m Normal cert-manager.io certificaterequest/odh-observability-webhook-cert-1 Certificate request has been approved by cert-manager.io 72m Normal Requested certificate/odh-observability-webhook-cert Created new CertificateRequest resource "odh-observability-webhook-cert-1" 72m Normal ScalingReplicaSet deployment/odh-observability Scaled up replica set odh-observability-584f99c788 from 0 to 1 72m Normal Issuing certificate/odh-observability-webhook-cert The certificate has been successfully issued 72m Warning BadConfig certificaterequest/odh-observability-webhook-cert-1 Certificate will be issued with an empty Issuer DN, which contravenes RFC 5280 and could break some strict clients 72m Warning FailedMount pod/odh-observability-584f99c788-k7579 MountVolume.SetUp failed for volume "webhook-certs" : secret "odh-observability-webhook-cert" not found 72m Normal Generated certificate/odh-observability-webhook-cert Stored new private key in temporary Secret resource "odh-observability-webhook-cert-l8ppf" 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal SuccessfulCreate replicaset/odh-observability-584f99c788 Created pod: odh-observability-584f99c788-k7579 72m Normal Issuing certificate/odh-observability-webhook-cert Issuing certificate as Secret does not exist 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal CertificateIssued certificaterequest/odh-observability-webhook-cert-1 Certificate fetched from issuer successfully 72m Normal Pulling pod/odh-observability-584f99c788-k7579 Pulling image "quay.io/opendatahub/odh-observability@sha256:03689e71fb8879ea696366b83d2a4727eb76e696baf8c2be2b18121c0534db4e" 72m Normal AddedInterface pod/odh-observability-584f99c788-k7579 Add eth0 [10.134.0.28/23] from ovn-kubernetes 72m Normal Created pod/odh-observability-584f99c788-k7579 Container created 72m Normal Started pod/odh-observability-584f99c788-k7579 Container started 72m Normal Pulled pod/odh-observability-584f99c788-k7579 Successfully pulled image "quay.io/opendatahub/odh-observability@sha256:03689e71fb8879ea696366b83d2a4727eb76e696baf8c2be2b18121c0534db4e" in 2.253s (2.253s including waiting). Image size: 154660917 bytes. 72m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-5cfb77bf8c Created pod: kserve-module-controller-manager-5cfb77bf8c-hv6k2 72m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-5cfb77bf8c from 0 to 1 72m Normal AddedInterface pod/kserve-module-controller-manager-5cfb77bf8c-hv6k2 Add eth0 [10.132.0.35/23] from ovn-kubernetes 72m Normal Pulling pod/kserve-module-controller-manager-5cfb77bf8c-hv6k2 Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 72m Normal Pulled pod/kserve-module-controller-manager-5cfb77bf8c-hv6k2 Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 1.36s (1.36s including waiting). Image size: 195729000 bytes. 72m Normal Started pod/kserve-module-controller-manager-5cfb77bf8c-hv6k2 Container started 72m Normal Created pod/kserve-module-controller-manager-5cfb77bf8c-hv6k2 Container created 71m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-5cfb77bf8c-hv6k2_9658a9ee-b6ea-4c21-84ac-b6ce30b686b5 became leader 71m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 71m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-vg6k7 71m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 71m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-nxcfv 71m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 71m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-vg6k7 Add eth0 [10.134.0.30/23] from ovn-kubernetes 71m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-zgr4b 71m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-nxcfv Add eth0 [10.134.0.31/23] from ovn-kubernetes 71m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-nxcfv Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 71m Normal Pulling pod/odh-model-controller-f87fcf56c-5fmhl Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 71m Normal AddedInterface pod/model-serving-api-75bd89fdfc-zgr4b Add eth0 [10.134.0.32/23] from ovn-kubernetes 71m Normal Pulling pod/model-serving-api-75bd89fdfc-zgr4b Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 71m Normal AddedInterface pod/odh-model-controller-f87fcf56c-5fmhl Add eth0 [10.134.0.33/23] from ovn-kubernetes 71m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-f87fcf56c from 0 to 1 71m Normal Pulling pod/kserve-controller-manager-776cc66d47-vg6k7 Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 71m Normal SuccessfulCreate replicaset/odh-model-controller-f87fcf56c Created pod: odh-model-controller-f87fcf56c-5fmhl 71m Normal Started pod/model-serving-api-75bd89fdfc-zgr4b Container started 71m Normal Created pod/model-serving-api-75bd89fdfc-zgr4b Container created 71m Normal Pulled pod/model-serving-api-75bd89fdfc-zgr4b Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 3.035s (3.035s including waiting). Image size: 184925761 bytes. 71m Normal Started pod/odh-model-controller-f87fcf56c-5fmhl Container started 71m Normal Pulled pod/kserve-controller-manager-776cc66d47-vg6k7 Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 4.239s (4.239s including waiting). Image size: 247676581 bytes. 71m Normal Created pod/odh-model-controller-f87fcf56c-5fmhl Container created 71m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-nxcfv Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 4.337s (4.337s including waiting). Image size: 235100981 bytes. 71m Normal Pulled pod/odh-model-controller-f87fcf56c-5fmhl Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 4.179s (4.179s including waiting). Image size: 236755567 bytes. 71m Normal Created pod/kserve-controller-manager-776cc66d47-vg6k7 Container created 71m Normal Started pod/kserve-controller-manager-776cc66d47-vg6k7 Container started 71m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-nxcfv Container started 71m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-nxcfv Container created 71m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-f87fcf56c-5fmhl_6b297b7d-2465-43f8-bd73-d21076ffb9db became leader 71m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-vg6k7_27031e83-704d-48b4-95b6-0d7a7fef9029 became leader 71m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-nxcfv_91a79b4f-449a-44e4-b39d-7b0966ff57a7 became leader 67m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-577f596567 from 0 to 1 67m Normal SuccessfulCreate replicaset/maas-controller-577f596567 Created pod: maas-controller-577f596567-h5692 67m Normal SuccessfulCreate replicaset/maas-controller-575c7b5c5b Created pod: maas-controller-575c7b5c5b-lh44m 67m Normal AddedInterface pod/maas-controller-577f596567-h5692 Add eth0 [10.132.0.45/23] from ovn-kubernetes 67m Normal Pulling pod/maas-controller-575c7b5c5b-lh44m Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 67m Normal AddedInterface pod/maas-controller-575c7b5c5b-lh44m Add eth0 [10.132.0.46/23] from ovn-kubernetes 67m Normal Pulling pod/maas-controller-577f596567-h5692 Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 67m Normal Created pod/maas-controller-577f596567-h5692 Container created 67m Normal Started pod/maas-controller-577f596567-h5692 Container started 67m Normal Pulled pod/maas-controller-577f596567-h5692 Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 2.681s (2.681s including waiting). Image size: 199691795 bytes. 67m Normal Pulled pod/maas-controller-575c7b5c5b-lh44m Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 2.51s (2.51s including waiting). Image size: 199691795 bytes. 67m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-577f596567-h5692_16801e4f-eb23-406a-bf01-5bbcbe9fcf84 became leader 67m Normal Created pod/maas-controller-575c7b5c5b-lh44m Container created 67m Normal Started pod/maas-controller-575c7b5c5b-lh44m Container started 67m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-7d94f55f4b from 0 to 1 67m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-7d94f55f4b Created pod: usage-logs-tenancy-proxy-7d94f55f4b-4f74l 67m Warning FailedMount pod/usage-logs-tenancy-proxy-7d94f55f4b-4f74l MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found 67m Warning FailedMount pod/usage-logs-tenancy-proxy-7d94f55f4b-4f74l MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 67m Normal Pulling pod/usage-logs-tenancy-proxy-7d94f55f4b-4f74l Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 67m Normal AddedInterface pod/usage-logs-tenancy-proxy-7d94f55f4b-4f74l Add eth0 [10.132.0.47/23] from ovn-kubernetes 67m Normal Killing pod/maas-controller-577f596567-h5692 Stopping container manager 67m Normal SuccessfulDelete replicaset/maas-controller-577f596567 Deleted pod: maas-controller-577f596567-h5692 67m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-577f596567 from 1 to 0 67m Normal Created pod/usage-logs-tenancy-proxy-7d94f55f4b-4f74l Container created 67m Normal Started pod/usage-logs-tenancy-proxy-7d94f55f4b-4f74l Container started 67m Normal Pulled pod/usage-logs-tenancy-proxy-7d94f55f4b-4f74l Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 10.688s (10.688s including waiting). Image size: 1089502687 bytes. 67m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-575c7b5c5b-lh44m_15d85ffd-c9e3-4a0e-af3c-f8d28d41625b became leader 67m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-vt287 67m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 67m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-vt287 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 67m Normal Created pod/llmisvc-controller-manager-6466c554d8-vt287 Container created 67m Normal Started pod/llmisvc-controller-manager-6466c554d8-vt287 Container started 67m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-vt287 Add eth0 [10.134.0.46/23] from ovn-kubernetes 67m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-vt287 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 346ms (346ms including waiting). Image size: 235100981 bytes. 66m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-nxcfv 66m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 66m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-nxcfv Stopping container manager 66m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-vt287_f53f7cd0-957a-4740-8ad7-62fd2ec80f64 became leader 65m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-6cf7bcbfb6 from 0 to 1 65m Normal SuccessfulCreate replicaset/ai-gateway-controller-6cf7bcbfb6 Created pod: ai-gateway-controller-6cf7bcbfb6-s4pl8 65m Normal AddedInterface pod/ai-gateway-controller-6cf7bcbfb6-s4pl8 Add eth0 [10.133.0.24/23] from ovn-kubernetes 65m Normal Pulling pod/ai-gateway-controller-6cf7bcbfb6-s4pl8 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" 65m Normal Started pod/ai-gateway-controller-6cf7bcbfb6-s4pl8 Container started 65m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-6cf7bcbfb6-s4pl8_b49afe8c-550e-4f23-b52c-362fead32fee became leader 65m Normal Created pod/ai-gateway-controller-6cf7bcbfb6-s4pl8 Container created 65m Normal Pulled pod/ai-gateway-controller-6cf7bcbfb6-s4pl8 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" in 8.987s (8.987s including waiting). Image size: 183930301 bytes. 65m Normal Pulling pod/ai-gateway-controller-6d48fc876f-4g6d7 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" 65m Normal SuccessfulCreate replicaset/ai-gateway-controller-6d48fc876f Created pod: ai-gateway-controller-6d48fc876f-4g6d7 65m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-6d48fc876f from 0 to 1 65m Normal AddedInterface pod/ai-gateway-controller-6d48fc876f-4g6d7 Add eth0 [10.133.0.28/23] from ovn-kubernetes 65m Normal Started pod/ai-gateway-controller-6d48fc876f-4g6d7 Container started 65m Normal Created pod/ai-gateway-controller-6d48fc876f-4g6d7 Container created 65m Normal Pulled pod/ai-gateway-controller-6d48fc876f-4g6d7 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" in 416ms (416ms including waiting). Image size: 183930301 bytes. 65m Normal Killing pod/ai-gateway-controller-6cf7bcbfb6-s4pl8 Stopping container manager 65m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-6cf7bcbfb6 from 1 to 0 65m Normal SuccessfulDelete replicaset/ai-gateway-controller-6cf7bcbfb6 Deleted pod: ai-gateway-controller-6cf7bcbfb6-s4pl8 64m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-6d48fc876f-4g6d7_c9e453d8-ca2c-402a-829a-18788b66c344 became leader 57m Normal Pulling pod/test-curl-8965-9edf49 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 57m Normal AddedInterface pod/test-curl-8965-9edf49 Add eth0 [10.132.0.87/23] from ovn-kubernetes 57m Normal Created pod/test-curl-8965-9edf49 Container created 57m Normal Started pod/test-curl-8965-9edf49 Container started 57m Normal Pulled pod/test-curl-8965-9edf49 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.336s (2.336s including waiting). Image size: 106100986 bytes. 57m Normal AddedInterface pod/test-curl-8965-cbe3f3 Add eth0 [10.132.0.88/23] from ovn-kubernetes 57m Normal Created pod/test-curl-8965-cbe3f3 Container created 57m Normal Pulled pod/test-curl-8965-cbe3f3 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 57m Normal Started pod/test-curl-8965-cbe3f3 Container started 57m Normal Killing pod/test-curl-8965-9edf49 Stopping container test-curl-8965-9edf49 57m Normal Killing pod/test-curl-8965-cbe3f3 Stopping container test-curl-8965-cbe3f3 51m Normal AddedInterface pod/test-curl-8968-1aa1d0 Add eth0 [10.132.0.122/23] from ovn-kubernetes 51m Normal Pulled pod/test-curl-8968-1aa1d0 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal Created pod/test-curl-8968-1aa1d0 Container created 51m Normal Started pod/test-curl-8968-1aa1d0 Container started 51m Normal Started pod/test-curl-8968-42007b Container started 51m Normal Killing pod/test-curl-8968-1aa1d0 Stopping container test-curl-8968-1aa1d0 51m Normal Created pod/test-curl-8968-42007b Container created 51m Normal Pulled pod/test-curl-8968-42007b Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/test-curl-8968-42007b Add eth0 [10.132.0.124/23] from ovn-kubernetes 51m Normal Created pod/test-curl-8968-efdfc3 Container created 51m Normal AddedInterface pod/test-curl-8968-efdfc3 Add eth0 [10.132.0.125/23] from ovn-kubernetes 51m Normal Killing pod/test-curl-8968-42007b Stopping container test-curl-8968-42007b 51m Normal Pulled pod/test-curl-8968-efdfc3 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal Started pod/test-curl-8968-efdfc3 Container started 51m Normal AddedInterface pod/test-curl-8968-f1788c Add eth0 [10.132.0.126/23] from ovn-kubernetes 51m Normal Started pod/test-curl-8968-f1788c Container started 51m Normal Killing pod/test-curl-8968-efdfc3 Stopping container test-curl-8968-efdfc3 51m Normal Created pod/test-curl-8968-f1788c Container created 51m Normal Pulled pod/test-curl-8968-f1788c Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal Killing pod/test-curl-8968-f1788c Stopping container test-curl-8968-f1788c 51m Normal Started pod/test-curl-8962-2f0f36 Container started 51m Normal AddedInterface pod/test-curl-8962-2f0f36 Add eth0 [10.132.0.129/23] from ovn-kubernetes 51m Normal Created pod/test-curl-8962-2f0f36 Container created 51m Normal Pulled pod/test-curl-8962-2f0f36 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal Killing pod/test-curl-8962-2f0f36 Stopping container test-curl-8962-2f0f36 47m Normal Pulling pod/test-curl-8962-44aa17 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 47m Normal AddedInterface pod/test-curl-8962-44aa17 Add eth0 [10.134.0.114/23] from ovn-kubernetes 47m Normal Created pod/test-curl-8962-44aa17 Container created 47m Normal Started pod/test-curl-8962-44aa17 Container started 47m Normal Pulled pod/test-curl-8962-44aa17 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 948ms (948ms including waiting). Image size: 106100986 bytes. 47m Normal Killing pod/test-curl-8962-44aa17 Stopping container test-curl-8962-44aa17 36m Normal AddedInterface pod/test-curl-8959-424b98 Add eth0 [10.134.0.149/23] from ovn-kubernetes 36m Normal Pulled pod/test-curl-8959-424b98 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 36m Normal Created pod/test-curl-8959-424b98 Container created 36m Normal Started pod/test-curl-8959-424b98 Container started 36m Normal Killing pod/test-curl-8959-424b98 Stopping container test-curl-8959-424b98 33m Normal Started pod/test-curl-8956-25e9f6 Container started 33m Normal Created pod/test-curl-8956-25e9f6 Container created 33m Normal Pulled pod/test-curl-8956-25e9f6 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/test-curl-8956-25e9f6 Add eth0 [10.134.0.154/23] from ovn-kubernetes 33m Normal AddedInterface pod/test-curl-8956-01d266 Add eth0 [10.134.0.156/23] from ovn-kubernetes 33m Normal Killing pod/test-curl-8956-25e9f6 Stopping container test-curl-8956-25e9f6 33m Normal Started pod/test-curl-8956-01d266 Container started 33m Normal Created pod/test-curl-8956-01d266 Container created 33m Normal Pulled pod/test-curl-8956-01d266 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 33m Normal Killing pod/test-curl-8956-01d266 Stopping container test-curl-8956-01d266 19m Normal Killing pod/maas-controller-575c7b5c5b-lh44m Stopping container manager 19m Normal SuccessfulDelete replicaset/maas-controller-575c7b5c5b Deleted pod: maas-controller-575c7b5c5b-lh44m 19m Normal AddedInterface pod/maas-controller-575c7b5c5b-f64sn Add eth0 [10.132.0.233/23] from ovn-kubernetes 19m Normal Pulling pod/maas-controller-575c7b5c5b-f64sn Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 19m Normal SuccessfulCreate replicaset/maas-controller-575c7b5c5b Created pod: maas-controller-575c7b5c5b-f64sn 19m Normal Pulled pod/maas-controller-575c7b5c5b-f64sn Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 479ms (479ms including waiting). Image size: 199691795 bytes. 19m Normal Created pod/maas-controller-575c7b5c5b-f64sn Container created 19m Normal Started pod/maas-controller-575c7b5c5b-f64sn Container started 19m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-575c7b5c5b-f64sn_a99d2660-fa01-4b44-99d5-a5084b765e93 became leader 18m Normal SuccessfulDelete replicaset/maas-controller-575c7b5c5b Deleted pod: maas-controller-575c7b5c5b-f64sn 18m Normal Killing pod/maas-controller-575c7b5c5b-f64sn Stopping container manager 18m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-575c7b5c5b from 1 to 0 18m Normal SuccessfulCreate replicaset/maas-controller-575c7b5c5b Created pod: maas-controller-575c7b5c5b-p8bxg 18m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-575c7b5c5b from 0 to 1 18m Normal Pulled pod/maas-controller-575c7b5c5b-p8bxg Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 348ms (348ms including waiting). Image size: 199691795 bytes. 18m Normal Pulling pod/maas-controller-575c7b5c5b-p8bxg Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 18m Normal AddedInterface pod/maas-controller-575c7b5c5b-p8bxg Add eth0 [10.132.0.235/23] from ovn-kubernetes 18m Normal Created pod/maas-controller-575c7b5c5b-p8bxg Container created 18m Normal Started pod/maas-controller-575c7b5c5b-p8bxg Container started 18m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-575c7b5c5b-p8bxg_d1456f53-93a8-41b9-bbf3-0650264a0695 became leader