=== events in opendatahub (2026-11-03T19:55:56Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 67m Normal Scheduled pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Successfully assigned opendatahub/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx to ip-10-0-139-179.us-west-2.compute.internal 67m Normal Scheduled pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Successfully assigned opendatahub/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s to ip-10-0-133-236.us-west-2.compute.internal 61m Normal Scheduled pod/usage-logs-tenancy-proxy-7d94f55f4b-msvrv Successfully assigned opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-msvrv to ip-10-0-139-179.us-west-2.compute.internal 46m Normal Scheduled pod/test-curl-8878-ae49a6 Successfully assigned opendatahub/test-curl-8878-ae49a6 to ip-10-0-133-236.us-west-2.compute.internal 46m Normal Scheduled pod/test-curl-8878-aca6be Successfully assigned opendatahub/test-curl-8878-aca6be to ip-10-0-133-236.us-west-2.compute.internal 39m Normal Scheduled pod/test-curl-8875-a22504 Successfully assigned opendatahub/test-curl-8875-a22504 to ip-10-0-139-179.us-west-2.compute.internal 39m Normal Scheduled pod/test-curl-8875-996e09 Successfully assigned opendatahub/test-curl-8875-996e09 to ip-10-0-139-179.us-west-2.compute.internal 39m Normal Scheduled pod/test-curl-8875-920945 Successfully assigned opendatahub/test-curl-8875-920945 to ip-10-0-139-179.us-west-2.compute.internal 65m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-qwrfw Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-qwrfw to ip-10-0-133-236.us-west-2.compute.internal 38m Normal Scheduled pod/test-curl-8863-f6d127 Successfully assigned opendatahub/test-curl-8863-f6d127 to ip-10-0-139-179.us-west-2.compute.internal 58m Normal Scheduled pod/ai-gateway-controller-7d4c5c4549-hsf4r Successfully assigned opendatahub/ai-gateway-controller-7d4c5c4549-hsf4r to ip-10-0-139-179.us-west-2.compute.internal 32m Normal Scheduled pod/test-curl-8863-0020d9 Successfully assigned opendatahub/test-curl-8863-0020d9 to ip-10-0-133-236.us-west-2.compute.internal 65m Normal Scheduled pod/odh-model-controller-f87fcf56c-h2qzd Successfully assigned opendatahub/odh-model-controller-f87fcf56c-h2qzd to ip-10-0-137-192.us-west-2.compute.internal 66m Normal Scheduled pod/odh-observability-584f99c788-r8rjd Successfully assigned opendatahub/odh-observability-584f99c788-r8rjd to ip-10-0-133-236.us-west-2.compute.internal 58m Normal Scheduled pod/ai-gateway-controller-7489558957-b4tmh Successfully assigned opendatahub/ai-gateway-controller-7489558957-b4tmh to ip-10-0-139-179.us-west-2.compute.internal 66m Normal Scheduled pod/kserve-module-controller-manager-5cfb77bf8c-2sbkn Successfully assigned opendatahub/kserve-module-controller-manager-5cfb77bf8c-2sbkn to ip-10-0-133-236.us-west-2.compute.internal 39m Normal Scheduled pod/test-curl-8875-8e8798 Successfully assigned opendatahub/test-curl-8875-8e8798 to ip-10-0-139-179.us-west-2.compute.internal 16m Normal Scheduled pod/maas-controller-bbcd964c6-ztfsd Successfully assigned opendatahub/maas-controller-bbcd964c6-ztfsd to ip-10-0-139-179.us-west-2.compute.internal 17m Normal Scheduled pod/maas-controller-bbcd964c6-76gws Successfully assigned opendatahub/maas-controller-bbcd964c6-76gws to ip-10-0-139-179.us-west-2.compute.internal 67m Normal Scheduled pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Successfully assigned opendatahub/opendatahub-operator-controller-manager-5ff9d8584c-fcklg to ip-10-0-137-192.us-west-2.compute.internal 61m Normal Scheduled pod/maas-controller-577f596567-ldvsg Successfully assigned opendatahub/maas-controller-577f596567-ldvsg to ip-10-0-139-179.us-west-2.compute.internal 60m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-fvh8n Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-fvh8n to ip-10-0-133-236.us-west-2.compute.internal 65m Normal Scheduled pod/kserve-controller-manager-776cc66d47-xsfxn Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-xsfxn to ip-10-0-133-236.us-west-2.compute.internal 65m Normal Scheduled pod/model-serving-api-75bd89fdfc-tx2vn Successfully assigned opendatahub/model-serving-api-75bd89fdfc-tx2vn to ip-10-0-137-192.us-west-2.compute.internal 61m Normal Scheduled pod/maas-controller-bbcd964c6-q7xqh Successfully assigned opendatahub/maas-controller-bbcd964c6-q7xqh to ip-10-0-139-179.us-west-2.compute.internal 67m Normal RequirementsNotMet clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 one or more requirements couldn't be found 67m Normal RequirementsUnknown clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 requirements not yet checked 67m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-5ff9d8584c Created pod: opendatahub-operator-controller-manager-5ff9d8584c-pdm9s 67m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-5ff9d8584c Created pod: opendatahub-operator-controller-manager-5ff9d8584c-pkhrx 67m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-5ff9d8584c Created pod: opendatahub-operator-controller-manager-5ff9d8584c-fcklg 67m Normal ScalingReplicaSet deployment/opendatahub-operator-controller-manager Scaled up replica set opendatahub-operator-controller-manager-5ff9d8584c from 0 to 3 67m Normal AllRequirementsMet clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 all requirements found, attempting install 67m Normal Pulling pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 67m Normal InstallSucceeded clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 waiting for install components to report healthy 67m Normal InstallWaiting clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability. 67m Normal AddedInterface pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Add eth0 [10.134.0.16/23] from ovn-kubernetes 67m Normal Pulling pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 67m Normal AddedInterface pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Add eth0 [10.133.0.20/23] from ovn-kubernetes 67m Normal Pulling pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 67m Normal AddedInterface pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Add eth0 [10.132.0.36/23] from ovn-kubernetes 67m Normal Created pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Container created 67m Normal Pulled pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.593s (3.593s including waiting). Image size: 273709574 bytes. 67m Normal Started pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Container started 67m Normal Pulling pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 67m Normal Started pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Container started 67m Normal Created pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Container created 67m Normal Pulled pod/opendatahub-operator-controller-manager-5ff9d8584c-pdm9s Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 916ms (916ms including waiting). Image size: 273709574 bytes. 67m Normal Started pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Container started 67m Normal LeaderElection lease/07ed84f7.opendatahub.io opendatahub-operator-controller-manager-5ff9d8584c-pdm9s_bd8943e9-0cd3-49f4-862b-96e54daad181 became leader 67m Normal Pulled pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 6.816s (6.816s including waiting). Image size: 273709574 bytes. 67m Normal Created pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Container created 67m Normal Pulling pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 67m Normal Started pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Container started 67m Normal Created pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Container created 67m Normal Pulled pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 8.364s (8.364s including waiting). Image size: 273709574 bytes. 67m Normal Started pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Container started 67m Normal Pulling pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 67m Normal Created pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Container created 67m Normal Pulled pod/opendatahub-operator-controller-manager-5ff9d8584c-pkhrx Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 928ms (928ms including waiting). Image size: 273709574 bytes. 67m Normal Created pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Container created 67m Normal Pulled pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 852ms (853ms including waiting). Image size: 273709574 bytes. 67m Normal Started pod/opendatahub-operator-controller-manager-5ff9d8584c-fcklg Container started 66m Normal InstallSucceeded clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 install strategy completed with no errors 66m Normal Requested certificate/odh-observability-webhook-cert Created new CertificateRequest resource "odh-observability-webhook-cert-1" 66m Normal CertificateIssued certificaterequest/odh-observability-webhook-cert-1 Certificate fetched from issuer successfully 66m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 66m Normal ScalingReplicaSet deployment/odh-observability Scaled up replica set odh-observability-584f99c788 from 0 to 1 66m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 66m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 66m Normal Generated certificate/odh-observability-webhook-cert Stored new private key in temporary Secret resource "odh-observability-webhook-cert-cwlrj" 66m Normal SuccessfulCreate replicaset/odh-observability-584f99c788 Created pod: odh-observability-584f99c788-r8rjd 66m Normal Issuing certificate/odh-observability-webhook-cert Issuing certificate as Secret does not exist 66m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 66m Warning BadConfig certificaterequest/odh-observability-webhook-cert-1 Certificate will be issued with an empty Issuer DN, which contravenes RFC 5280 and could break some strict clients 66m Normal cert-manager.io certificaterequest/odh-observability-webhook-cert-1 Certificate request has been approved by cert-manager.io 66m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 66m Warning FailedMount pod/odh-observability-584f99c788-r8rjd MountVolume.SetUp failed for volume "webhook-certs" : secret "odh-observability-webhook-cert" not found 66m Normal Issuing certificate/odh-observability-webhook-cert The certificate has been successfully issued 66m Normal AddedInterface pod/odh-observability-584f99c788-r8rjd Add eth0 [10.133.0.23/23] from ovn-kubernetes 66m Normal Pulling pod/odh-observability-584f99c788-r8rjd Pulling image "quay.io/opendatahub/odh-observability@sha256:03689e71fb8879ea696366b83d2a4727eb76e696baf8c2be2b18121c0534db4e" 66m Normal Created pod/odh-observability-584f99c788-r8rjd Container created 66m Normal Started pod/odh-observability-584f99c788-r8rjd Container started 66m Normal Pulled pod/odh-observability-584f99c788-r8rjd Successfully pulled image "quay.io/opendatahub/odh-observability@sha256:03689e71fb8879ea696366b83d2a4727eb76e696baf8c2be2b18121c0534db4e" in 4.102s (4.102s including waiting). Image size: 154660917 bytes. 66m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-5cfb77bf8c Created pod: kserve-module-controller-manager-5cfb77bf8c-2sbkn 66m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-5cfb77bf8c from 0 to 1 66m Normal AddedInterface pod/kserve-module-controller-manager-5cfb77bf8c-2sbkn Add eth0 [10.133.0.24/23] from ovn-kubernetes 66m Normal Pulling pod/kserve-module-controller-manager-5cfb77bf8c-2sbkn Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 66m Normal Pulled pod/kserve-module-controller-manager-5cfb77bf8c-2sbkn Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.777s (2.777s including waiting). Image size: 195729000 bytes. 66m Normal Created pod/kserve-module-controller-manager-5cfb77bf8c-2sbkn Container created 66m Normal Started pod/kserve-module-controller-manager-5cfb77bf8c-2sbkn Container started 66m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-5cfb77bf8c-2sbkn_edfa5611-77cf-47ad-b019-c0a7632ee532 became leader 65m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 65m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-f87fcf56c from 0 to 1 65m Normal AddedInterface pod/odh-model-controller-f87fcf56c-h2qzd Add eth0 [10.134.0.20/23] from ovn-kubernetes 65m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-xsfxn Add eth0 [10.133.0.25/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 65m Normal Pulling pod/kserve-controller-manager-776cc66d47-xsfxn Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 65m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-qwrfw 65m Normal SuccessfulCreate replicaset/odh-model-controller-f87fcf56c Created pod: odh-model-controller-f87fcf56c-h2qzd 65m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-qwrfw Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 65m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-xsfxn 65m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-tx2vn 65m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 65m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-qwrfw Add eth0 [10.133.0.26/23] from ovn-kubernetes 65m Normal AddedInterface pod/model-serving-api-75bd89fdfc-tx2vn Add eth0 [10.134.0.19/23] from ovn-kubernetes 65m Normal Pulling pod/model-serving-api-75bd89fdfc-tx2vn Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 65m Normal Pulling pod/odh-model-controller-f87fcf56c-h2qzd Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 65m Normal Pulled pod/model-serving-api-75bd89fdfc-tx2vn Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.36s (2.36s including waiting). Image size: 184925761 bytes. 65m Normal Created pod/model-serving-api-75bd89fdfc-tx2vn Container created 65m Normal Started pod/model-serving-api-75bd89fdfc-tx2vn Container started 65m Normal Started pod/odh-model-controller-f87fcf56c-h2qzd Container started 65m Normal Created pod/odh-model-controller-f87fcf56c-h2qzd Container created 65m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-qwrfw Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.522s (3.522s including waiting). Image size: 235100981 bytes. 65m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-qwrfw Container started 65m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-qwrfw Container created 65m Normal Pulled pod/odh-model-controller-f87fcf56c-h2qzd Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.067s (3.067s including waiting). Image size: 236755567 bytes. 65m Normal Pulled pod/kserve-controller-manager-776cc66d47-xsfxn Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 4.44s (4.441s including waiting). Image size: 247676581 bytes. 65m Normal Created pod/kserve-controller-manager-776cc66d47-xsfxn Container created 65m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-f87fcf56c-h2qzd_774ff302-709f-4b29-a3f9-92b535e4a402 became leader 65m Normal Started pod/kserve-controller-manager-776cc66d47-xsfxn Container started 65m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-xsfxn_9cbccb33-e3d9-443e-8456-5d35085993e5 became leader 65m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-qwrfw_d03d7524-72c1-4057-a570-02ac5cc7333b became leader 61m Normal SuccessfulCreate replicaset/maas-controller-577f596567 Created pod: maas-controller-577f596567-ldvsg 61m Normal SuccessfulCreate replicaset/maas-controller-bbcd964c6 Created pod: maas-controller-bbcd964c6-q7xqh 61m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-577f596567 from 0 to 1 61m Normal AddedInterface pod/maas-controller-577f596567-ldvsg Add eth0 [10.132.0.50/23] from ovn-kubernetes 61m Normal Pulling pod/maas-controller-577f596567-ldvsg Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 61m Normal AddedInterface pod/maas-controller-bbcd964c6-q7xqh Add eth0 [10.132.0.51/23] from ovn-kubernetes 61m Normal Pulling pod/maas-controller-bbcd964c6-q7xqh Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 61m Normal Started pod/maas-controller-577f596567-ldvsg Container started 61m Normal Pulled pod/maas-controller-bbcd964c6-q7xqh Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 5.106s (5.106s including waiting). Image size: 199691795 bytes. 61m Normal Created pod/maas-controller-bbcd964c6-q7xqh Container created 61m Normal Pulled pod/maas-controller-577f596567-ldvsg Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 5.562s (5.562s including waiting). Image size: 199691795 bytes. 61m Normal Created pod/maas-controller-577f596567-ldvsg Container created 61m Normal Started pod/maas-controller-bbcd964c6-q7xqh Container started 61m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-577f596567-ldvsg_64b00402-55fb-477e-80dc-259a4af56f45 became leader 61m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-7d94f55f4b from 0 to 1 61m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-7d94f55f4b Created pod: usage-logs-tenancy-proxy-7d94f55f4b-msvrv 61m Warning FailedMount pod/usage-logs-tenancy-proxy-7d94f55f4b-msvrv MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found 61m Warning FailedMount pod/usage-logs-tenancy-proxy-7d94f55f4b-msvrv MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 61m Normal Pulling pod/usage-logs-tenancy-proxy-7d94f55f4b-msvrv Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 61m Normal AddedInterface pod/usage-logs-tenancy-proxy-7d94f55f4b-msvrv Add eth0 [10.132.0.52/23] from ovn-kubernetes 61m Normal SuccessfulDelete replicaset/maas-controller-577f596567 Deleted pod: maas-controller-577f596567-ldvsg 61m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-577f596567 from 1 to 0 61m Normal Killing pod/maas-controller-577f596567-ldvsg Stopping container manager 61m Normal Created pod/usage-logs-tenancy-proxy-7d94f55f4b-msvrv Container created 61m Normal Pulled pod/usage-logs-tenancy-proxy-7d94f55f4b-msvrv Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 14.123s (14.123s including waiting). Image size: 1089502687 bytes. 61m Normal Started pod/usage-logs-tenancy-proxy-7d94f55f4b-msvrv Container started 60m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-bbcd964c6-q7xqh_0209c84d-f5cf-4fcf-a191-5d60a169ea49 became leader 60m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-fvh8n Add eth0 [10.133.0.34/23] from ovn-kubernetes 60m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 60m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-fvh8n Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 60m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-fvh8n 60m Normal Created pod/llmisvc-controller-manager-6466c554d8-fvh8n Container created 60m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-fvh8n Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 721ms (721ms including waiting). Image size: 235100981 bytes. 60m Normal Started pod/llmisvc-controller-manager-6466c554d8-fvh8n Container started 60m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-qwrfw Stopping container manager 60m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-qwrfw 60m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 59m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-fvh8n_05c8c528-5c00-42c8-9af8-202c216011b9 became leader 58m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-7489558957 from 0 to 1 58m Normal SuccessfulCreate replicaset/ai-gateway-controller-7489558957 Created pod: ai-gateway-controller-7489558957-b4tmh 58m Normal Pulling pod/ai-gateway-controller-7489558957-b4tmh Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-174" 58m Normal AddedInterface pod/ai-gateway-controller-7489558957-b4tmh Add eth0 [10.132.0.59/23] from ovn-kubernetes 58m Normal Started pod/ai-gateway-controller-7489558957-b4tmh Container started 58m Normal Created pod/ai-gateway-controller-7489558957-b4tmh Container created 58m Normal Pulled pod/ai-gateway-controller-7489558957-b4tmh Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-174" in 5.908s (5.908s including waiting). Image size: 183794449 bytes. 58m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-7489558957-b4tmh_a3da1899-3515-4493-92ce-1a42dbce1431 became leader 58m Normal AddedInterface pod/ai-gateway-controller-7d4c5c4549-hsf4r Add eth0 [10.132.0.61/23] from ovn-kubernetes 58m Normal Pulling pod/ai-gateway-controller-7d4c5c4549-hsf4r Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-174" 58m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-7d4c5c4549 from 0 to 1 58m Normal SuccessfulCreate replicaset/ai-gateway-controller-7d4c5c4549 Created pod: ai-gateway-controller-7d4c5c4549-hsf4r 58m Normal Pulled pod/ai-gateway-controller-7d4c5c4549-hsf4r Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-174" in 954ms (954ms including waiting). Image size: 183794449 bytes. 58m Normal Created pod/ai-gateway-controller-7d4c5c4549-hsf4r Container created 58m Normal Started pod/ai-gateway-controller-7d4c5c4549-hsf4r Container started 58m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-7489558957 from 1 to 0 58m Normal SuccessfulDelete replicaset/ai-gateway-controller-7489558957 Deleted pod: ai-gateway-controller-7489558957-b4tmh 58m Normal Killing pod/ai-gateway-controller-7489558957-b4tmh Stopping container manager 58m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-7d4c5c4549-hsf4r_294f5cb6-0116-4737-9c2c-543fc52e638f became leader 46m Normal AddedInterface pod/test-curl-8878-ae49a6 Add eth0 [10.133.0.76/23] from ovn-kubernetes 46m Normal Pulling pod/test-curl-8878-ae49a6 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 46m Normal Pulled pod/test-curl-8878-ae49a6 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 3.464s (3.464s including waiting). Image size: 106100986 bytes. 46m Normal Created pod/test-curl-8878-ae49a6 Container created 46m Normal Started pod/test-curl-8878-ae49a6 Container started 46m Normal Killing pod/test-curl-8878-ae49a6 Stopping container test-curl-8878-ae49a6 46m Normal Created pod/test-curl-8878-aca6be Container created 46m Normal AddedInterface pod/test-curl-8878-aca6be Add eth0 [10.133.0.77/23] from ovn-kubernetes 46m Normal Started pod/test-curl-8878-aca6be Container started 46m Normal Pulled pod/test-curl-8878-aca6be Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 46m Normal Killing pod/test-curl-8878-aca6be Stopping container test-curl-8878-aca6be 46m Warning Unhealthy pod/model-serving-api-75bd89fdfc-tx2vn Readiness probe failed: Get "https://10.134.0.19:8443/readyz": context deadline exceeded 39m Normal Pulling pod/test-curl-8875-920945 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 39m Normal AddedInterface pod/test-curl-8875-920945 Add eth0 [10.132.0.129/23] from ovn-kubernetes 39m Normal Started pod/test-curl-8875-920945 Container started 39m Normal Created pod/test-curl-8875-920945 Container created 39m Normal Pulled pod/test-curl-8875-920945 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 3.258s (3.258s including waiting). Image size: 106100986 bytes. 39m Normal Killing pod/test-curl-8875-920945 Stopping container test-curl-8875-920945 39m Normal Created pod/test-curl-8875-8e8798 Container created 39m Normal Started pod/test-curl-8875-8e8798 Container started 39m Normal Pulled pod/test-curl-8875-8e8798 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/test-curl-8875-8e8798 Add eth0 [10.132.0.130/23] from ovn-kubernetes 39m Normal Started pod/test-curl-8875-996e09 Container started 39m Normal Killing pod/test-curl-8875-8e8798 Stopping container test-curl-8875-8e8798 39m Normal AddedInterface pod/test-curl-8875-996e09 Add eth0 [10.132.0.131/23] from ovn-kubernetes 39m Normal Pulled pod/test-curl-8875-996e09 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 39m Normal Created pod/test-curl-8875-996e09 Container created 39m Normal Killing pod/test-curl-8875-996e09 Stopping container test-curl-8875-996e09 39m Normal Started pod/test-curl-8875-a22504 Container started 39m Normal Created pod/test-curl-8875-a22504 Container created 39m Normal Pulled pod/test-curl-8875-a22504 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/test-curl-8875-a22504 Add eth0 [10.132.0.132/23] from ovn-kubernetes 39m Normal Killing pod/test-curl-8875-a22504 Stopping container test-curl-8875-a22504 39m Normal AddedInterface pod/test-curl-8863-f6d127 Add eth0 [10.132.0.140/23] from ovn-kubernetes 38m Normal Created pod/test-curl-8863-f6d127 Container created 38m Normal Started pod/test-curl-8863-f6d127 Container started 38m Normal Pulled pod/test-curl-8863-f6d127 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 38m Normal Killing pod/test-curl-8863-f6d127 Stopping container test-curl-8863-f6d127 32m Normal Pulled pod/test-curl-8863-0020d9 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/test-curl-8863-0020d9 Add eth0 [10.133.0.112/23] from ovn-kubernetes 32m Normal Created pod/test-curl-8863-0020d9 Container created 32m Normal Started pod/test-curl-8863-0020d9 Container started 32m Normal Killing pod/test-curl-8863-0020d9 Stopping container test-curl-8863-0020d9 17m Normal Killing pod/maas-controller-bbcd964c6-q7xqh Stopping container manager 17m Normal SuccessfulDelete replicaset/maas-controller-bbcd964c6 Deleted pod: maas-controller-bbcd964c6-q7xqh 17m Normal SuccessfulCreate replicaset/maas-controller-bbcd964c6 Created pod: maas-controller-bbcd964c6-76gws 17m Normal AddedInterface pod/maas-controller-bbcd964c6-76gws Add eth0 [10.132.0.164/23] from ovn-kubernetes 17m Normal Pulling pod/maas-controller-bbcd964c6-76gws Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 17m Normal Started pod/maas-controller-bbcd964c6-76gws Container started 17m Normal Created pod/maas-controller-bbcd964c6-76gws Container created 17m Normal Pulled pod/maas-controller-bbcd964c6-76gws Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 869ms (869ms including waiting). Image size: 199691795 bytes. 17m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-bbcd964c6-76gws_e97cc434-e516-4b0e-86ac-4aaa5f4f1eb3 became leader 17m Normal Killing pod/maas-controller-bbcd964c6-76gws Stopping container manager 17m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-bbcd964c6 from 1 to 0 17m Normal SuccessfulDelete replicaset/maas-controller-bbcd964c6 Deleted pod: maas-controller-bbcd964c6-76gws 17m Normal SuccessfulCreate replicaset/maas-controller-bbcd964c6 Created pod: maas-controller-bbcd964c6-ztfsd 17m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-bbcd964c6 from 0 to 1 16m Normal AddedInterface pod/maas-controller-bbcd964c6-ztfsd Add eth0 [10.132.0.166/23] from ovn-kubernetes 16m Normal Pulling pod/maas-controller-bbcd964c6-ztfsd Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 16m Normal Pulled pod/maas-controller-bbcd964c6-ztfsd Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 818ms (818ms including waiting). Image size: 199691795 bytes. 16m Normal Created pod/maas-controller-bbcd964c6-ztfsd Container created 16m Normal Started pod/maas-controller-bbcd964c6-ztfsd Container started 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-bbcd964c6-ztfsd_853f1d6f-114f-459d-8385-683a6bcb509d became leader