=== events in opendatahub (2026-10-30T20:24:03Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 72m Normal Scheduled pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Successfully assigned opendatahub/opendatahub-operator-controller-manager-8589ff54bc-ldgdt to ip-10-0-135-68.ec2.internal 71m Normal Scheduled pod/kserve-controller-manager-776cc66d47-lhwvl Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-lhwvl to ip-10-0-130-217.ec2.internal 67m Normal Scheduled pod/usage-logs-tenancy-proxy-7d94f55f4b-vz8vl Successfully assigned opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-vz8vl to ip-10-0-140-235.ec2.internal 29m Normal Scheduled pod/test-curl-9247-cdc9f2 Successfully assigned opendatahub/test-curl-9247-cdc9f2 to ip-10-0-135-68.ec2.internal 29m Normal Scheduled pod/test-curl-9247-78f35d Successfully assigned opendatahub/test-curl-9247-78f35d to ip-10-0-135-68.ec2.internal 46m Normal Scheduled pod/test-curl-9244-dd08be Successfully assigned opendatahub/test-curl-9244-dd08be to ip-10-0-135-68.ec2.internal 49m Normal Scheduled pod/test-curl-9244-29d7a8 Successfully assigned opendatahub/test-curl-9244-29d7a8 to ip-10-0-135-68.ec2.internal 55m Normal Scheduled pod/test-curl-9241-b9dcc0 Successfully assigned opendatahub/test-curl-9241-b9dcc0 to ip-10-0-135-68.ec2.internal 55m Normal Scheduled pod/test-curl-9241-7445af Successfully assigned opendatahub/test-curl-9241-7445af to ip-10-0-135-68.ec2.internal 51m Normal Scheduled pod/test-curl-9238-7fac8c Successfully assigned opendatahub/test-curl-9238-7fac8c to ip-10-0-135-68.ec2.internal 64m Normal Scheduled pod/ai-gateway-controller-84dd667bd6-hz7tl Successfully assigned opendatahub/ai-gateway-controller-84dd667bd6-hz7tl to ip-10-0-140-235.ec2.internal 51m Normal Scheduled pod/test-curl-9238-7e9019 Successfully assigned opendatahub/test-curl-9238-7e9019 to ip-10-0-135-68.ec2.internal 51m Normal Scheduled pod/test-curl-9238-5e0631 Successfully assigned opendatahub/test-curl-9238-5e0631 to ip-10-0-135-68.ec2.internal 51m Normal Scheduled pod/test-curl-9238-0c9587 Successfully assigned opendatahub/test-curl-9238-0c9587 to ip-10-0-135-68.ec2.internal 30m Normal Scheduled pod/test-curl-9235-f174a8 Successfully assigned opendatahub/test-curl-9235-f174a8 to ip-10-0-135-68.ec2.internal 67m Normal Scheduled pod/maas-controller-577f596567-vfgdk Successfully assigned opendatahub/maas-controller-577f596567-vfgdk to ip-10-0-140-235.ec2.internal 66m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-c757k Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-c757k to ip-10-0-130-217.ec2.internal 67m Normal Scheduled pod/maas-controller-65cb454cc4-j4fbd Successfully assigned opendatahub/maas-controller-65cb454cc4-j4fbd to ip-10-0-140-235.ec2.internal 72m Normal Scheduled pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Successfully assigned opendatahub/opendatahub-operator-controller-manager-8589ff54bc-ch865 to ip-10-0-140-235.ec2.internal 71m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-sjv8b Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-sjv8b to ip-10-0-130-217.ec2.internal 64m Normal Scheduled pod/ai-gateway-controller-5d9cd99f45-g2nv5 Successfully assigned opendatahub/ai-gateway-controller-5d9cd99f45-g2nv5 to ip-10-0-140-235.ec2.internal 72m Normal Scheduled pod/kserve-module-controller-manager-5cfb77bf8c-7dx5m Successfully assigned opendatahub/kserve-module-controller-manager-5cfb77bf8c-7dx5m to ip-10-0-130-217.ec2.internal 18m Normal Scheduled pod/maas-controller-65cb454cc4-5lnl7 Successfully assigned opendatahub/maas-controller-65cb454cc4-5lnl7 to ip-10-0-140-235.ec2.internal 72m Normal Scheduled pod/odh-observability-584f99c788-564b7 Successfully assigned opendatahub/odh-observability-584f99c788-564b7 to ip-10-0-140-235.ec2.internal 71m Normal Scheduled pod/odh-model-controller-f87fcf56c-gqrwf Successfully assigned opendatahub/odh-model-controller-f87fcf56c-gqrwf to ip-10-0-130-217.ec2.internal 71m Normal Scheduled pod/model-serving-api-75bd89fdfc-m6f5w Successfully assigned opendatahub/model-serving-api-75bd89fdfc-m6f5w to ip-10-0-130-217.ec2.internal 72m Normal Scheduled pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Successfully assigned opendatahub/opendatahub-operator-controller-manager-8589ff54bc-8jpmq to ip-10-0-130-217.ec2.internal 19m Normal Scheduled pod/maas-controller-65cb454cc4-q7tpn Successfully assigned opendatahub/maas-controller-65cb454cc4-q7tpn to ip-10-0-140-235.ec2.internal 72m Normal RequirementsNotMet clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 one or more requirements couldn't be found 72m Normal AllRequirementsMet clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 all requirements found, attempting install 72m Normal RequirementsUnknown clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 requirements not yet checked 72m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-8589ff54bc Created pod: opendatahub-operator-controller-manager-8589ff54bc-ldgdt 72m Normal AddedInterface pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Add eth0 [10.134.0.19/23] from ovn-kubernetes 72m Normal AddedInterface pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Add eth0 [10.133.0.25/23] from ovn-kubernetes 72m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-8589ff54bc Created pod: opendatahub-operator-controller-manager-8589ff54bc-8jpmq 72m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-8589ff54bc Created pod: opendatahub-operator-controller-manager-8589ff54bc-ch865 72m Normal Pulling pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal ScalingReplicaSet deployment/opendatahub-operator-controller-manager Scaled up replica set opendatahub-operator-controller-manager-8589ff54bc from 0 to 3 72m Normal AddedInterface pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Add eth0 [10.132.0.30/23] from ovn-kubernetes 72m Normal Pulling pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal Pulling pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal InstallSucceeded clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 waiting for install components to report healthy 72m Normal InstallWaiting clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability. 72m Normal Started pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Container started 72m Normal Pulled pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.789s (2.789s including waiting). Image size: 273709574 bytes. 72m Normal Pulled pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.8s (2.8s including waiting). Image size: 273709574 bytes. 72m Normal Pulled pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.817s (2.817s including waiting). Image size: 273709574 bytes. 72m Normal Created pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Container created 72m Normal Created pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Container created 72m Normal Created pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Container created 72m Normal Started pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Container started 72m Normal Started pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Container started 72m Normal Pulling pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal Pulling pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal Pulling pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 72m Normal Created pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Container created 72m Normal Pulled pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 327ms (327ms including waiting). Image size: 273709574 bytes. 72m Normal Created pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Container created 72m Normal Pulled pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 297ms (297ms including waiting). Image size: 273709574 bytes. 72m Normal Started pod/opendatahub-operator-controller-manager-8589ff54bc-ch865 Container started 72m Normal Pulled pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 380ms (380ms including waiting). Image size: 273709574 bytes. 72m Normal Started pod/opendatahub-operator-controller-manager-8589ff54bc-ldgdt Container started 72m Normal Created pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Container created 72m Normal Started pod/opendatahub-operator-controller-manager-8589ff54bc-8jpmq Container started 72m Normal LeaderElection lease/07ed84f7.opendatahub.io opendatahub-operator-controller-manager-8589ff54bc-ldgdt_c51498d2-9775-4f04-ad1c-a43c47c686b2 became leader 72m Normal InstallSucceeded clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 install strategy completed with no errors 72m Normal Generated certificate/odh-observability-webhook-cert Stored new private key in temporary Secret resource "odh-observability-webhook-cert-qx7wd" 72m Normal Issuing certificate/odh-observability-webhook-cert Issuing certificate as Secret does not exist 72m Normal SuccessfulCreate replicaset/odh-observability-584f99c788 Created pod: odh-observability-584f99c788-564b7 72m Normal Issuing certificate/odh-observability-webhook-cert The certificate has been successfully issued 72m Normal Requested certificate/odh-observability-webhook-cert Created new CertificateRequest resource "odh-observability-webhook-cert-1" 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal ScalingReplicaSet deployment/odh-observability Scaled up replica set odh-observability-584f99c788 from 0 to 1 72m Normal cert-manager.io certificaterequest/odh-observability-webhook-cert-1 Certificate request has been approved by cert-manager.io 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal CertificateIssued certificaterequest/odh-observability-webhook-cert-1 Certificate fetched from issuer successfully 72m Warning BadConfig certificaterequest/odh-observability-webhook-cert-1 Certificate will be issued with an empty Issuer DN, which contravenes RFC 5280 and could break some strict clients 72m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 72m Normal Pulling pod/odh-observability-584f99c788-564b7 Pulling image "quay.io/opendatahub/odh-observability@sha256:03689e71fb8879ea696366b83d2a4727eb76e696baf8c2be2b18121c0534db4e" 72m Normal AddedInterface pod/odh-observability-584f99c788-564b7 Add eth0 [10.132.0.31/23] from ovn-kubernetes 72m Normal Pulled pod/odh-observability-584f99c788-564b7 Successfully pulled image "quay.io/opendatahub/odh-observability@sha256:03689e71fb8879ea696366b83d2a4727eb76e696baf8c2be2b18121c0534db4e" in 2.606s (2.606s including waiting). Image size: 154660917 bytes. 72m Normal Created pod/odh-observability-584f99c788-564b7 Container created 72m Normal Started pod/odh-observability-584f99c788-564b7 Container started 72m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-5cfb77bf8c Created pod: kserve-module-controller-manager-5cfb77bf8c-7dx5m 72m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-5cfb77bf8c from 0 to 1 72m Normal AddedInterface pod/kserve-module-controller-manager-5cfb77bf8c-7dx5m Add eth0 [10.133.0.27/23] from ovn-kubernetes 72m Normal Pulling pod/kserve-module-controller-manager-5cfb77bf8c-7dx5m Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 72m Normal Started pod/kserve-module-controller-manager-5cfb77bf8c-7dx5m Container started 72m Normal Created pod/kserve-module-controller-manager-5cfb77bf8c-7dx5m Container created 72m Normal Pulled pod/kserve-module-controller-manager-5cfb77bf8c-7dx5m Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.33s (2.33s including waiting). Image size: 195729000 bytes. 71m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-5cfb77bf8c-7dx5m_7045e1fd-24ee-40a5-8071-a7b1bcf96ae6 became leader 71m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-lhwvl 71m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-lhwvl Add eth0 [10.133.0.29/23] from ovn-kubernetes 71m Normal SuccessfulCreate replicaset/odh-model-controller-f87fcf56c Created pod: odh-model-controller-f87fcf56c-gqrwf 71m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 71m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 71m Normal AddedInterface pod/model-serving-api-75bd89fdfc-m6f5w Add eth0 [10.133.0.31/23] from ovn-kubernetes 71m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-f87fcf56c from 0 to 1 71m Normal Pulling pod/model-serving-api-75bd89fdfc-m6f5w Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 71m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-sjv8b Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 71m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 71m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-sjv8b 71m Normal AddedInterface pod/odh-model-controller-f87fcf56c-gqrwf Add eth0 [10.133.0.32/23] from ovn-kubernetes 71m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-sjv8b Add eth0 [10.133.0.30/23] from ovn-kubernetes 71m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-m6f5w 71m Normal Pulling pod/odh-model-controller-f87fcf56c-gqrwf Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 71m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-sjv8b Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.016s (3.016s including waiting). Image size: 235100981 bytes. 71m Normal Pulled pod/model-serving-api-75bd89fdfc-m6f5w Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.622s (2.622s including waiting). Image size: 184925761 bytes. 71m Normal Created pod/model-serving-api-75bd89fdfc-m6f5w Container created 71m Normal Started pod/model-serving-api-75bd89fdfc-m6f5w Container started 71m Normal Pulled pod/kserve-controller-manager-776cc66d47-lhwvl Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.931s (2.931s including waiting). Image size: 247676581 bytes. 71m Normal Created pod/odh-model-controller-f87fcf56c-gqrwf Container created 71m Normal Pulled pod/odh-model-controller-f87fcf56c-gqrwf Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.415s (3.415s including waiting). Image size: 236755567 bytes. 71m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-sjv8b Container started 71m Normal Started pod/odh-model-controller-f87fcf56c-gqrwf Container started 71m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-sjv8b Container created 71m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-f87fcf56c-gqrwf_11c3662c-d8b7-431e-a175-667532570b60 became leader 71m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-sjv8b_d0ae7028-0359-4ba6-b897-048b74ee83d6 became leader 71m Normal Pulled pod/kserve-controller-manager-776cc66d47-lhwvl Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 266ms (266ms including waiting). Image size: 247676581 bytes. 70m Warning BackOff pod/kserve-controller-manager-776cc66d47-lhwvl Back-off restarting failed container manager in pod kserve-controller-manager-776cc66d47-lhwvl_opendatahub(3ec07eb2-2af5-4d2b-ae2d-d905c57085f8) 70m Normal Pulled pod/kserve-controller-manager-776cc66d47-lhwvl Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 235ms (235ms including waiting). Image size: 247676581 bytes. 70m Normal Created pod/kserve-controller-manager-776cc66d47-lhwvl Container created 70m Normal Started pod/kserve-controller-manager-776cc66d47-lhwvl Container started 70m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-lhwvl_94ee4a20-e216-47c3-a561-28a60d744b5a became leader 70m Normal Pulling pod/kserve-controller-manager-776cc66d47-lhwvl Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 67m Normal SuccessfulCreate replicaset/maas-controller-65cb454cc4 Created pod: maas-controller-65cb454cc4-j4fbd 67m Normal SuccessfulCreate replicaset/maas-controller-577f596567 Created pod: maas-controller-577f596567-vfgdk 67m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-577f596567 from 0 to 1 67m Normal AddedInterface pod/maas-controller-577f596567-vfgdk Add eth0 [10.132.0.41/23] from ovn-kubernetes 67m Normal AddedInterface pod/maas-controller-65cb454cc4-j4fbd Add eth0 [10.132.0.42/23] from ovn-kubernetes 67m Normal Pulling pod/maas-controller-577f596567-vfgdk Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 67m Normal Pulling pod/maas-controller-65cb454cc4-j4fbd Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 67m Normal Started pod/maas-controller-577f596567-vfgdk Container started 67m Normal Created pod/maas-controller-577f596567-vfgdk Container created 67m Normal Pulled pod/maas-controller-577f596567-vfgdk Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 2.647s (2.647s including waiting). Image size: 199691795 bytes. 67m Normal Pulled pod/maas-controller-65cb454cc4-j4fbd Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 2.634s (2.634s including waiting). Image size: 199691795 bytes. 67m Normal Started pod/maas-controller-65cb454cc4-j4fbd Container started 67m Normal Created pod/maas-controller-65cb454cc4-j4fbd Container created 67m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-577f596567-vfgdk_4da2d003-9c9c-4b95-a9b6-6b1dbfebb025 became leader 67m Warning FailedMount pod/usage-logs-tenancy-proxy-7d94f55f4b-vz8vl MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 67m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-7d94f55f4b Created pod: usage-logs-tenancy-proxy-7d94f55f4b-vz8vl 67m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-7d94f55f4b from 0 to 1 67m Normal Pulling pod/usage-logs-tenancy-proxy-7d94f55f4b-vz8vl Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 67m Normal AddedInterface pod/usage-logs-tenancy-proxy-7d94f55f4b-vz8vl Add eth0 [10.132.0.43/23] from ovn-kubernetes 66m Normal SuccessfulDelete replicaset/maas-controller-577f596567 Deleted pod: maas-controller-577f596567-vfgdk 66m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-577f596567 from 1 to 0 66m Normal Killing pod/maas-controller-577f596567-vfgdk Stopping container manager 66m Normal Started pod/usage-logs-tenancy-proxy-7d94f55f4b-vz8vl Container started 66m Normal Pulled pod/usage-logs-tenancy-proxy-7d94f55f4b-vz8vl Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 10.911s (10.911s including waiting). Image size: 1089502687 bytes. 66m Normal Created pod/usage-logs-tenancy-proxy-7d94f55f4b-vz8vl Container created 66m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-65cb454cc4-j4fbd_7d064fa9-e52e-4570-b0a0-07a08bb4f4dd became leader 66m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 66m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-c757k 66m Normal Started pod/llmisvc-controller-manager-6466c554d8-c757k Container started 66m Normal Created pod/llmisvc-controller-manager-6466c554d8-c757k Container created 66m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-c757k Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 66m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-c757k Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 349ms (349ms including waiting). Image size: 235100981 bytes. 66m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-c757k Add eth0 [10.133.0.38/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 65m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-sjv8b Stopping container manager 65m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-sjv8b 65m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-c757k_e4ce33d7-2592-48c6-940d-3184e6253319 became leader 64m Normal Pulling pod/ai-gateway-controller-5d9cd99f45-g2nv5 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-191" 64m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-5d9cd99f45 from 0 to 1 64m Normal AddedInterface pod/ai-gateway-controller-5d9cd99f45-g2nv5 Add eth0 [10.132.0.50/23] from ovn-kubernetes 64m Normal SuccessfulCreate replicaset/ai-gateway-controller-5d9cd99f45 Created pod: ai-gateway-controller-5d9cd99f45-g2nv5 64m Normal Pulled pod/ai-gateway-controller-5d9cd99f45-g2nv5 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-191" in 2.859s (2.859s including waiting). Image size: 183908778 bytes. 64m Normal Created pod/ai-gateway-controller-5d9cd99f45-g2nv5 Container created 64m Normal Started pod/ai-gateway-controller-5d9cd99f45-g2nv5 Container started 64m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-5d9cd99f45-g2nv5_70be9eba-a6b7-4683-a317-d55c5aeca3bc became leader 64m Normal SuccessfulCreate replicaset/ai-gateway-controller-84dd667bd6 Created pod: ai-gateway-controller-84dd667bd6-hz7tl 64m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-84dd667bd6 from 0 to 1 64m Normal Pulled pod/ai-gateway-controller-84dd667bd6-hz7tl Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-191" in 369ms (369ms including waiting). Image size: 183908778 bytes. 64m Normal AddedInterface pod/ai-gateway-controller-84dd667bd6-hz7tl Add eth0 [10.132.0.53/23] from ovn-kubernetes 64m Normal Pulling pod/ai-gateway-controller-84dd667bd6-hz7tl Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-191" 64m Normal Started pod/ai-gateway-controller-84dd667bd6-hz7tl Container started 64m Normal Created pod/ai-gateway-controller-84dd667bd6-hz7tl Container created 64m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-5d9cd99f45 from 1 to 0 64m Normal SuccessfulDelete replicaset/ai-gateway-controller-5d9cd99f45 Deleted pod: ai-gateway-controller-5d9cd99f45-g2nv5 64m Normal Killing pod/ai-gateway-controller-5d9cd99f45-g2nv5 Stopping container manager 63m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-84dd667bd6-hz7tl_ee0877b6-3147-4284-b2b7-c53fa035b666 became leader 55m Normal Pulling pod/test-curl-9241-b9dcc0 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 55m Normal AddedInterface pod/test-curl-9241-b9dcc0 Add eth0 [10.134.0.106/23] from ovn-kubernetes 55m Normal Pulled pod/test-curl-9241-b9dcc0 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 3.013s (3.013s including waiting). Image size: 106100986 bytes. 55m Normal Started pod/test-curl-9241-b9dcc0 Container started 55m Normal Created pod/test-curl-9241-b9dcc0 Container created 55m Normal AddedInterface pod/test-curl-9241-7445af Add eth0 [10.134.0.109/23] from ovn-kubernetes 55m Normal Pulled pod/test-curl-9241-7445af Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 55m Normal Killing pod/test-curl-9241-b9dcc0 Stopping container test-curl-9241-b9dcc0 55m Normal Created pod/test-curl-9241-7445af Container created 55m Normal Started pod/test-curl-9241-7445af Container started 55m Normal Killing pod/test-curl-9241-7445af Stopping container test-curl-9241-7445af 51m Normal AddedInterface pod/test-curl-9238-7fac8c Add eth0 [10.134.0.137/23] from ovn-kubernetes 51m Normal Pulled pod/test-curl-9238-7fac8c Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal Created pod/test-curl-9238-7fac8c Container created 51m Normal Started pod/test-curl-9238-7fac8c Container started 51m Normal Pulled pod/test-curl-9238-0c9587 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/test-curl-9238-0c9587 Add eth0 [10.134.0.139/23] from ovn-kubernetes 51m Normal Started pod/test-curl-9238-0c9587 Container started 51m Normal Created pod/test-curl-9238-0c9587 Container created 51m Normal Killing pod/test-curl-9238-7fac8c Stopping container test-curl-9238-7fac8c 51m Normal AddedInterface pod/test-curl-9238-7e9019 Add eth0 [10.134.0.140/23] from ovn-kubernetes 51m Normal Pulled pod/test-curl-9238-7e9019 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal Created pod/test-curl-9238-7e9019 Container created 51m Normal Started pod/test-curl-9238-7e9019 Container started 51m Normal Killing pod/test-curl-9238-0c9587 Stopping container test-curl-9238-0c9587 51m Normal Pulled pod/test-curl-9238-5e0631 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/test-curl-9238-5e0631 Add eth0 [10.134.0.141/23] from ovn-kubernetes 51m Normal Started pod/test-curl-9238-5e0631 Container started 51m Normal Created pod/test-curl-9238-5e0631 Container created 51m Normal Killing pod/test-curl-9238-7e9019 Stopping container test-curl-9238-7e9019 51m Normal Killing pod/test-curl-9238-5e0631 Stopping container test-curl-9238-5e0631 49m Normal Pulled pod/test-curl-9244-29d7a8 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/test-curl-9244-29d7a8 Add eth0 [10.134.0.155/23] from ovn-kubernetes 49m Normal Created pod/test-curl-9244-29d7a8 Container created 49m Normal Started pod/test-curl-9244-29d7a8 Container started 49m Normal Killing pod/test-curl-9244-29d7a8 Stopping container test-curl-9244-29d7a8 46m Normal Created pod/test-curl-9244-dd08be Container created 46m Normal Pulled pod/test-curl-9244-dd08be Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/test-curl-9244-dd08be Add eth0 [10.134.0.172/23] from ovn-kubernetes 46m Normal Started pod/test-curl-9244-dd08be Container started 46m Normal Killing pod/test-curl-9244-dd08be Stopping container test-curl-9244-dd08be 30m Normal AddedInterface pod/test-curl-9235-f174a8 Add eth0 [10.134.0.252/23] from ovn-kubernetes 30m Normal Pulled pod/test-curl-9235-f174a8 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 30m Normal Created pod/test-curl-9235-f174a8 Container created 30m Normal Started pod/test-curl-9235-f174a8 Container started 30m Normal Killing pod/test-curl-9235-f174a8 Stopping container test-curl-9235-f174a8 29m Normal AddedInterface pod/test-curl-9247-cdc9f2 Add eth0 [10.134.1.5/23] from ovn-kubernetes 29m Normal Pulled pod/test-curl-9247-cdc9f2 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 29m Normal Created pod/test-curl-9247-cdc9f2 Container created 29m Normal Started pod/test-curl-9247-cdc9f2 Container started 29m Normal Created pod/test-curl-9247-78f35d Container created 29m Normal Started pod/test-curl-9247-78f35d Container started 29m Normal Pulled pod/test-curl-9247-78f35d Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/test-curl-9247-78f35d Add eth0 [10.134.1.6/23] from ovn-kubernetes 29m Normal Killing pod/test-curl-9247-cdc9f2 Stopping container test-curl-9247-cdc9f2 29m Normal Killing pod/test-curl-9247-78f35d Stopping container test-curl-9247-78f35d 19m Normal Killing pod/maas-controller-65cb454cc4-j4fbd Stopping container manager 19m Normal SuccessfulDelete replicaset/maas-controller-65cb454cc4 Deleted pod: maas-controller-65cb454cc4-j4fbd 19m Normal Pulling pod/maas-controller-65cb454cc4-q7tpn Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 19m Normal AddedInterface pod/maas-controller-65cb454cc4-q7tpn Add eth0 [10.132.0.228/23] from ovn-kubernetes 19m Normal SuccessfulCreate replicaset/maas-controller-65cb454cc4 Created pod: maas-controller-65cb454cc4-q7tpn 19m Normal Started pod/maas-controller-65cb454cc4-q7tpn Container started 19m Normal Created pod/maas-controller-65cb454cc4-q7tpn Container created 19m Normal Pulled pod/maas-controller-65cb454cc4-q7tpn Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 420ms (420ms including waiting). Image size: 199691795 bytes. 18m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-65cb454cc4-q7tpn_e4cd59a6-45a1-4faa-b799-8ae9c59b9d85 became leader 18m Normal SuccessfulDelete replicaset/maas-controller-65cb454cc4 Deleted pod: maas-controller-65cb454cc4-q7tpn 18m Normal Killing pod/maas-controller-65cb454cc4-q7tpn Stopping container manager 18m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-65cb454cc4 from 1 to 0 18m Normal AddedInterface pod/maas-controller-65cb454cc4-5lnl7 Add eth0 [10.132.0.230/23] from ovn-kubernetes 18m Normal Pulling pod/maas-controller-65cb454cc4-5lnl7 Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 18m Normal Pulled pod/maas-controller-65cb454cc4-5lnl7 Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 317ms (317ms including waiting). Image size: 199691795 bytes. 18m Normal SuccessfulCreate replicaset/maas-controller-65cb454cc4 Created pod: maas-controller-65cb454cc4-5lnl7 18m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-65cb454cc4 from 0 to 1 18m Normal Created pod/maas-controller-65cb454cc4-5lnl7 Container created 18m Normal Started pod/maas-controller-65cb454cc4-5lnl7 Container started 18m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-65cb454cc4-5lnl7_ac21a721-245c-4ab2-8309-a3a3f222f241 became leader